CORESense provides a basic robots.txt file that can be edited from the Website Template Manager.
|
Step |
Action |
|
1 |
To view the current content of the file got the URL: http://www.companyname.com/robots.txt.The basic code is as follows. User-agent: 008 Disallow: / User-agent: * Crawl-delay: 120 Disallow: |
|
2 |
Go to Products > Channel Managers > Website Template Manager, and select your most recent website project. |
|
3 |
Click on the Dev tab and search for the tag, "robots.txt." |
|
4 |
The robots.txt template is there. Click on the file to make edits. |
|
5 |
Click on Save after editing and Preview. |
|
6 |
If it looks correct, Publish the site. Go to URL in Step 1 to verify file. |
© 2023 CORESense · info@coresense.com · 125 High Rock Avenue, Saratoga Springs, NY 12866