Blocking Pages from Search Engine Visibility Scans
If you do not want Search Engine Visibility to analyze your entire website, you can block specific pages using your Website's robots.txt file. This is useful if you have admin pages or forums that you don't want to optimize.
To prevent Search Engine Visibility from scanning a page, update your Website's robots.txt file with the following, where page is the specific page in your Website:
User-agent: trafficblazer
Disallow: /page.html
You can also block entire directories with the following, where directory is the name of the directory or folder you want to block:
User-agent: trafficblazer
Disallow: /directory/
For more information about building a robots.txt file for your Website, see Controlling Website Crawling Using Search Engine Visibility.
