COS-General

Can I block files in my file manager from being indexed by search engines?

Last updated: October 7, 2016

Available For:

Product: HubSpot Marketing
Subscription: Basic, Professional, & Enterprise

Yes. If you wish to block files in your file manager, such as a PDF document, from being indexed by search engines, you will need to select a connected subdomain for the file(s) and use the file URL to be blocked from crawlers.

The default file URL looks like this. Notice 'CDN' as the subdomain:
http://cdn2.hubspot.net/hub/your-HubID/file-xxxxxxxx-pdf/docs/marketing_offer_january_2016.pdf

Selecting a connected subdomain looks like this:
http://www.your-subdomain.com/hs-fs/hub/your-HubID/file-xxxxxxxx-pdf/docs/marketing_offer_january_2016.pdf

To select a connected subdomain:

  • Go to Content > File Manager
  • Locate the file you wish to block from crawlers > Click the gear > Get File URL

Get-File-Path-URL.png

  • Click the dropdown to select a connected subdomain. A connected subdomain will display your company's domain, such as www.your-company.com or info.your-company.org

  • You may use this new file URL containing your company's connected subdomain when blocking the file from being indexed by search engines via a Robots.txt file. To install a Robots.txt file, click here to learn more.

use-connected-subdomain-file-url.png