HubSpot uses web crawlers to provide you with up-to-date, real-time information in your account's analytics tools, and to display live previews of your pages. If you’re seeing a spike in visits from HubSpot crawlers in your Google Analytics data, you have a few options to get a clearer sense of your traffic:
-
Set up a robots.txt file to stop crawlers from accessing parts of your sites. All of HubSpot's crawlers are fully compliant with rules you set for your own domains. Setting up a robots.txt file will prevent HubSpot's crawlers (and others, including Google's) from crawling your pages. Learn more about setting up a robots.txt file.
-
Filter out the specific crawler user-agent in Google Analytics. HubSpot sources and other HubSpot reporting tools will automatically exclude our crawler traffic, but if you'd like to get a more accurate report of traffic in Google Analytics, follow the instructions to exclude the crawler name as a referrer.
-
Filter out all bot traffic from Google Analytics. If you're not certain what the exact name of the crawler that is causing increased traffic to your site, Google provides a checkbox for you to filter out all bot/crawler traffic in general. Learn more about filtering bot/crawler traffic.
