Email

How does an email A/B test work?

Last updated: February 7, 2017

Available For:

Marketing: Pro, Enterprise
Sales: N/A

The information noted below explains how this testing feature works. Please click here if you would like to learn how actually create an A/B test email.

Please note email A/B testing is only available for HubSpot Marketing Professional and Enterprise accounts.

1000 contacts+

  • Your total send list must include at least 1,000 contacts to run an A/B test. This is because we need at least that large a sample size to gather statistically significant A/B test results.
  • You can select any sample size to test and we will send the winning version to the remaining percentage of the recipient list. In the example below, HubSpot will send out your email to 30% of your recipients at first. 15% will receive version A, and 15% will receive version B. After the time to gather test results has expired, the winning version will be sent to the remaining 70% of your recipients.
AB Test Sample Size
  • Because your contacts will not always open and read your emails right away, you will need to specify a time limit on how long we'll gather results before choosing a winning version.
  • There are also settings to select a version to send to your contacts if the sample size is too low or if the test is inconclusive.

Fewer than 1000 contacts

  • If you have fewer than 1000 contacts in your send you can choose to run a 50/50 test instead.

  • Otherwise, we'll send the entire list to the version you specify in the "Version to send if sample too low" selector.

  • If you choose to run a 50/50 test, you don't need to pick a test duration or winning version metric.