This error may appear if the page is hosted on a HubSpot staging domain:
- If the URL contains hs-sites.com, this indicates that the page is still on the HubSpot staging domain, which is automatically prevented from being indexed by search engines in the robots.txt file. In order to build up the SEO for your website and pages, your content should be on a domain or subdomain that is part of your website.
- You will be able to see the total page views for pages on the HubSpot staging domain and the URLs are public to share; however, they are blocked from being crawled by search engines.
Another reason this error may appear is if the page is blocking our two crawlers:
- HubSpot uses two crawlers, for pages and for links. The user agent for each of the crawlers is:
- Pages: "HubSpot Crawler 1.0 http://www.hubspot.com/"
- Links: "HubSpot Links Crawler 1.0 http://www.hubspot.com/"
To allow for the HubSpot crawlers to crawl your page, you may need to modify the robots.txt file you have implemented or you may need to modify how you are blocking crawlers at the server level.
- How do I refresh the data in Page Performance?
- Why does Page Performance show a "301: Moved Permanently" error message when I try to refresh a page's data?
- Why does Page Performance show a "404: Not Found" error message when I try to refresh a page's data?
- Why does Page Performance show a "302: Object Moved" error message when I try to refresh a page's data?