If the URL contains hs-sites.com, the page is still on the HubSpot staging domain, which is automatically prevented from being indexed by search engines in the robots.txt file. In order to build up the SEO for your website and pages, your content should be on a domain or subdomain that is part of your website. You will be able to see the total page views for pages on the HubSpot staging domain and the URLs are public to share; however, they are blocked from being crawled by search engines.
Another reason this error may appear is if the page is blocking our two crawlers. HubSpot uses two different crawlers for pages and links. The user agent for each of the crawlers is:
- Pages: "HubSpot Crawler 1.0 http://www.hubspot.com/"
- Links: "HubSpot Links Crawler 1.0 http://www.hubspot.com/"
To allow the HubSpot crawlers to crawl your page, you may need to modify the robots.txt file you have implemented or you may need to modify how you are blocking crawlers at the server-level.