Reports

Why are there HubSpot crawlers in my Google Analytics data?

Last updated: November 19, 2018

free

HubSpot uses web crawlers to provide you with up-to-date, real-time information in your account's analytics tools, and to display live previews of your pages. If you’re seeing a spike in visits from HubSpot crawlers in your Google Analytics data, you have a few options to get a clearer sense of your traffic: 

  1. Set up a robots.txt file to stop crawlers from accessing parts of your sites. All of HubSpot's crawlers are fully compliant with rules you set for your own domains. Setting up a robots.txt file will prevent HubSpot's crawlers (and others, including Google's) from crawling your pages. Learn more about setting up a robots.txt file.

  2. Filter out the specific crawler user-agent in Google Analytics. HubSpot sources and other HubSpot reporting tools will automatically exclude our crawler traffic, but if you'd like to get a more accurate report of traffic in Google Analytics, follow the instructions to exclude the crawler name as a referrer

  3. Filter out all bot traffic from Google Analytics. If you're not certain what the exact name of the crawler that is causing increased traffic to your site, Google provides a checkbox for you to filter out all bot/crawler traffic in general. Learn more about filtering bot/crawler traffic