You can customize the contents of the Robots.txt file for your website pages in the page publishing options for your domain.
What can you do with a robots.txt file for your subdomain?
- You can prevent thank you pages with content offerings from being indexed by search engines.
- You can prevent landing pages that are tied to email only campaigns from being indexed by search engines.
- You can prevent any pages containing duplicate content from being crawled.
1. Find your Robots.txt file
Go to Content > Content Settings.
Scroll down to the page publishing options to the Robots.txt section.
2. Customize the robots.txt section
3. Save your changes
Click on the Save changes button to finish updating your robots.txt file.
Next Article: How to test and publish your website pages