BetaRun A/B tests on marketing email subject lines

An A/B test measures engagement for different versions of the same email with a sample of your recipients. The best performing version is then sent to the rest of the recipients. 

While creating a marketing email, you can use Breeze, HubSpot's AI, to create an A/B test for the email's subject line. 

This article covers creating an A/B test for marketing email subject lines during the email creation process. Learn more about creating A/B tests for marketing email content. 

Please note: you can manage AI feature access in your AI settings and configure what data is shared. Review HubSpot's AI Trust FAQs and AI Model Cards for detailed information on AI security controls, data use, and compliance.

Before you get started

Before you begin working with this feature, make sure to fully understand what steps should be taken ahead of time, as well as the limitations of the feature and potential considerations of using it.

Understand requirements

• To generate A/B tested subject lines, you will need to turn on Give users access to generative AI tools and features in your AI settings.
• At least one segment must be selected in the Send to dropdown menu to perform an A/B test. 

Understand limitations and considerations

• Marketing emails can use either a generated A/B tested subject line or a customizable A/B test. If you want to run a customizable A/B test, remove the subject line A/B test first. 
• For best results, send the email to at least 1000 recipients. 

Add an A/B tested subject line to an email

1. In your HubSpot account, navigate to Marketing > Email.
2. Click the name of a marketing email, or click Create email in the top right.
3. In the email editor, click the inbox content module at the top of the editor. 

4. In the sidebar editor, enter a subject line in the Subject line field. 
5. Under your subject line, an alternative will be generated based on subject lines from other emails you've sent with high open rates. 
6. To use this generated option for A/B testing, click A/B test subject line. 

7. To proceed without A/B testing the subject line, click Dismiss. 
8. To edit settings for the A/B test before sending:

• • In the sidebar editor, click Manage A/B test. 

• • In the dialog box, select whether you'll send the email to a percentage of your contacts, then the winner to the rest, or send both versions equally to all contacts.
  • If you selected Send winner automatically, you can click to expand Adjust settings to set how the winner is determined, how long the test takes, and the test sample size. 
  • Click Save. 

7. To cancel the A/B test before sending the email: 
   • In the sidebar editor, click Manage A/B test. 
   • In the dialog box, click Remove test. 
8. Finalize and send the email. 

Analyze A/B test performance

After your email is sent, wait enough time to gather results. You can analyze the performance of your email variations in the email details:

1. In your HubSpot account, navigate to Marketing > Email. 
2. Click the name of your A/B email. The results of the A/B test will be summarized on the Performance page. The winning email version and metric will be highlighted in green.
3. In the upper left, click the Email version dropdown menu to analyze the email results for your email overall, or for either of the variations.