Skip to content
Email

A/B test your marketing email

Last updated: May 7, 2021

Applies to:

Marketing Hub Professional, Enterprise

An A/B test measures engagement for different versions of the same email with a sample of your recipients. After creating your marketing email, you'll create a version B to test how a single variation impacts email opens or clicks. Depending on how each version of your email performs over a period of time, the tool can pick a winning version to send to the rest of your recipients list.

Please note: you won't be able to create an A/B test with an email that is saved for automation for your workflows.

Create your A/B test emails

  • In your HubSpot account, navigate to Marketing > Email.

  • Click Create email. This will be version A of your A/B test.

  • In the top left of the editor, click test Run a test.
  • In the dialog box, enter a name for the variation B email, then click OK.
  • After creating both versions of your A/B test email, you can switch between the two at any time. In the upper left of the content editor, click the Variation A or Variation B tab.
  • While editing your variations, you can consider varying:

    • Offers: experiment with the medium of the offer. You might test an eBook versus a whitepaper or video.
    • Copy: experiment with the formatting and style of the content. You could test plain paragraphs versus bullet points or a longer block of text vs a shorter block of text.
    • Email sender: try sending the email from an employee's email address instead of a generic department address.
    • Image: try out different images to see how the conversion rate is influenced.
    • Subject line: play around with the length of the subject line or add personalization.
    • Whole email: the fastest way to achieve drastic results and produce a landing page that drives a lot of conversions is to test the entire email. Make iterations to the whole email that affect image placement, subject line, and its copy. Once you have a statistically significant result pointing to the variation that performed better, you can continue optimizing through smaller tweaks. 
  • Once you've finished editing your A and B email variations, click the Settings tab. In the A/B test section, you can configure your testing options:
    • A/B distribution: use the slider to decide what percentage of contacts will receive Variation A and Variation B. You can also set the slider to send Variation A to 50% and Variation B to 50%. This will send one email variation to half of the contacts in the list, and the other email to the other half of contacts. This lets you analyze the statistics of the two emails after the send to see which variation was more successful.
    • Winning metric: if you've opted to send variations A and B to a smaller group, then send the winning version to the remaining recipients, click the Winning metric dropdown menu to select the metric that determines how the winning variation will be chosen: Open rate, Click rate, or Click through rate.
    • Test duration: use the slider to set a time period to gather data before choosing a winner and sending the winning variation. If the results of the test are inconclusive after the test duration, HubSpot will automatically fallback to sending Variation A to the remaining recipients.

Please note:

  • You can cancel the winning send while the A/B test is still processing. In your HubSpot account, navigate to Marketing > Email, then click the name of the A/B email you want to cancel. In the upper right, click the Actions dropdown menu and select Cancel winning send.
  • You can run a full-split (50/50) A/B test using any sample size. All other sample splits require a list that includes at least 1,000 recipients, excluding any contacts that previously hard bounced or unsubscribed.

Send your A/B test email and gather results

After your email is sent, wait enough time to gather results. You can analyze the performance of your email variations in the email details:

  • In your HubSpot account, navigate to Marketing > Email. 
  • Click the name of your A/B email. The results of the A/B test will be summarized on the Performance page. The winning email version and metric will be highlighted in green.

  • In the upper left, click the Email version dropdown menu to analyze the email results for your email overall, or for either of the variations.

  • When analyzing the overall results or the performance of either variant, you can also click See details at the top of the page to review the original A/B test configuration.

email-details-a-b-test-configuration-1

Please note: the Test distribution percentage in the Email details panel refers to the combined sample split of both versions. For example, if you configured your test to send Variation A to 10% and Variation B to 10%, then the Test distribution percentage would be 20%.