Email

A/B test your marketing email

Last updated: January 14, 2019

Applies to:

Marketing Hub
marketing-pro-enterprise
Professional, Enterprise

An A/B test measures engagement for different versions of the same email with a sample of your recipients. After creating your marketing email, you'll create a version B to test how a single variation impacts email opens or clicks. Depending on how each version of your email performs over a period of time, the tool can pick a winning version to send to the rest of your recipients list.

Please note: you won't be able to create an A/B test with an email that is saved for automation for your workflows. Make sure your email is set immediately, or at a specific date and time.

 

Create your A/B test emails

  • In your HubSpot account, navigate to Marketing > Email.
  • Click Create email. This will be version A of your A/B test.
  • In the left sidebar, click the A/B icon, then click Create A/B test.

  • In the dialog box, enter a name for the variation B email, then click OK.

After creating both versions of your A/B test email, you can switch between the two at any time. In the upper left of the content editor, click the A or B tab.

HubSpot Help article screenshot

While editing your variations, you can consider varying:

  • Offers: experiment with the medium of the offer. You might test an eBook versus a whitepaper or video.
  • Copy: experiment with the formatting and style of the content. You could test plain paragraphs versus bullet points or a longer block of text vs a shorter block of text.
  • Email sender: try sending the email from an employee's email address instead of a generic department address.
  • Image: try out different images to see how the conversion rate is influenced.
  • Subject line: play around with the length of the subject line or add personalization.
  • Whole email: the fastest way to achieve drastic results and produce a landing page that drives a lot of conversions is to test the entire email. Make iterations to the whole email that affect image placement, subject line, and its copy. Once you have a statistically significant result pointing to the variation that performed better, you can continue optimizing through smaller tweaks.

 

Select the criteria for your A/B test

Once you've finished editing your A and B email variations, configure your testing options by clicking the A/B tab in the left sidebar.

Here, you can configure your testing options:

  • A/B distribution: use the slider to decide what percentage of contacts will receive Variation A and Variation B. You can also set the slider to send Variation A to 50% and Variation B to 50%. This will send one email variation to half of the contacts in the list, and the other email to the other half of contacts. This lets you analyze the statistics of the two emails after the send to see which variation was more successful.
  • Winning metric: if you've opted to send variations A and B to a smaller group, then send the winning version to the remaining recipients, click the Winning metric dropdown menu to select the metric that determines how the winning variation will be chosen: Opens by deliveredClicks by delivered, or Clicks by opens.
  • Test duration: use the slider to set a time period to gather data before choosing a winner and sending the winning variation. Your A/B test can run for up to 100 hours.
  • If the sample size is too low, send: if the final list size is too low (less than 1,000 contacts), the version selected in the dropdown menu will be the variation that is sent to the recipients.
  • If the results are inconclusive, send: if the results are inconclusive after the test duration has concluded, the version selected in the dropdown menu will be the variation that gets sent to the remaining recipients.

Please note: you can run a full-split (50/50) A/B test using any sample size. All other sample splits require a list that includes at least 1,000 recipients, excluding any contacts that previously hard bounced or unsubscribed.

 

Send your A/B test email and gather results

Once you've finished setting up the criteria for your A/B test, you're ready to send your emails. Click Send in the top-right corner of the email editor.

You can see the performance of your email variations by navigating to the email tool and clicking the name of your A/B email. The results of the A/B test will be summarized on the Performance page. The winning email version and metric will be highlighted in green.

In the upper left, click the Email version dropdown menu to analyze the email results for your total email send or for either of your email variations.