An A/B test measures engagement for different versions of the same email with a sample of your recipients. After creating your marketing email, you'll create a version B to test how a single variation impacts email opens or clicks. Depending on how each version of your email performs over a period of time, the tool can pick a winning version to send to the rest of your recipients list.
Please note: you won't be able to create an A/B test with an email that is saved for automation for your workflows.
Create your A/B test emails
In your HubSpot account, navigate to Marketing > Email.
Click Create email. This will be version A of your A/B test.
In the left sidebar, click the A/B icon, then click Create A/B test.
- In the dialog box, enter a name for the variation B email, then click OK. After creating both versions of your A/B test email, you can switch between the two at any time by clicking the A or B tab.
While editing your variations, you can consider varying:
- Offers: experiment with the medium of the offer. You might test an eBook versus a whitepaper or video.
- Copy: experiment with the formatting and style of the content. You could test plain paragraphs versus bullet points or a longer block of text vs a shorter block of text.
- Email sender: try sending the email from an employee's email address instead of a generic department address.
- Image: try out different images to see how the conversion rate is influenced.
- Subject line: play around with the length of the subject line or add personalization.
- Whole email: the fastest way to achieve drastic results and produce a landing page that drives a lot of conversions is to test the entire email. Make iterations to the whole email that affect image placement, subject line, and its copy. Once you have a statistically significant result pointing to the variation that performed better, you can continue optimizing through smaller tweaks.
Select the criteria for your A/B test
Once you've finished editing your A and B email variations, configure your testing options by clicking the A/B tab on the left side of the content editor.
Here, you can configure your testing options. Learn more about how to determine your A/B test time duration and distribution.
- A/B distribution: use the slider to decide what percentage of contacts will receive Variation A and Variation B. You can also set the slider to send Variation A to 50% and Variation B to 50%, then analyze the statistics of both versions after the send. You can run a full-split (50/50) A/B test using any sample size. All other sample splits require a list that includes at least 1,000 recipients, excluding any contacts that previously hard bounced or unsubscribed.
Please note: contacts who previously bounced or unsubscribed will be dropped from your test distribution, resulting in a small sample size.
- Winning metric: if you've opted to send variations A and B to a smaller group, then send the winning version to the remaining recipients, click the Winning metric dropdown menu to select the metric that determines how the winning variation will be chosen: Opens by delivered, Clicks by delivered, or Clicks by opens.
- Test duration: use the slider to set a time period to gather data before choosing a winner and sending the winning variation. Your A/B test can run for up to 100 hours.
- If the sample size is too low, send: if the final list size is too small (less than 1,000 contacts), only the version selected in this dropdown menu will be sent to the recipients.
- If the results are inconclusive, send: if the results for the winning metric are inconclusive after the test duration has ended, the version selected here will be sent to the remaining recipients. Be sure that your email is sent to a large enough sample size, and that your test duration runs long enough to gather results. Learn more about how statistical significance is determined in an A/B test email.
Please note: you can cancel the winning send while the A/B test is still processing. In your HubSpot account, navigate to Marketing > Email. Click the name of the A/B email you want to cancel. In the upper right, click the Actions dropdown menu and select Cancel winning send.
Send your A/B test email and gather results
- Click the Send or schedule tab.
- Select a specific time to schedule the A/B test email, or click Send in the upper right to send the email now.
After your email is sent, wait enough time to gather results. You can analyze the performance of your email variations in the email details:
- In your HubSpot account, navigate to Marketing > Email.
- Click the name of your A/B email. The results of the A/B test will be summarized on the Performance page. The winning email version and metric will be highlighted in green.
- In the upper left, click the Email version dropdown menu to analyze the email results for your email overall, or for either of the variations.