COS-General

Can I block files in my file manager from being indexed by search engines?

Last updated: February 24, 2017

Available For:

Marketing: Basic, Pro, Enterprise
Sales: N/A

Yes. If you wish to block an individual file in your file manager, such as a PDF document, from being indexed by search engines, you will need to select a connected subdomain for the file(s) and use the file URL to be blocked from crawlers.

The default file URL looks like this. Notice 'CDN' as the subdomain:
http://cdn2.hubspot.net/hub/your-HubID/file-xxxxxxxx-pdf/docs/marketing_offer_january_2016.pdf

Selecting a connected subdomain looks like this:
http://www.your-subdomain.com/hs-fs/hub/your-HubID/file-xxxxxxxx-pdf/docs/marketing_offer_january_2016.pdf

To select a connected subdomain for your file:

  • Navigate in HubSpot Marketing to Content > File Manager
  • Locate the file you wish to block from crawlers > click on the file name.
  • In the right sidebar, click the File URL dropdown. A connected subdomain will display your company's domain, such as www.your-company.com or info.your-company.org. Depending on the domains connected to HubSpot, you might have multiple options in this dropdown.

  • Select the file URL containing your company's connected subdomain > click Copy URL.

  • You can now use this URL when blocking the file from being indexed by search engines via a Robots.txt file. To learn how to install a Robots.txt file, click hereNote: If you have shared the file online using more than one connected subdomain included in the dropdown, copy the file URL for each connected subdomain to then add to your Robots.txt file.