Why were my A/B email test results inconclusive?

Last updated: October 17, 2018

Applies to:

Marketing Hub Professional, Enterprise

HubSpot performs a statistical test on your A/B test email. If neither email is statistically better, HubSpot will then send the version of the email chosen when creating the A/B test to the rest of the list. 

It's important you give your A/B test a decent amount of time to gather results, so the HubSpot email tool can have enough time to gather statistics and take actions on sending the winning variation. This will default to 1 hour when you create a new A/B test but it's recommended to extend this in order to allow enough time to gather helpful data.

The validity of the statistical test is based on three factors:

  • The overall difference in rates (opened, clicked, etc.)
  • The rates themselves need to be high enough to be significant.
  • The overall total sample size.

It is not possible to make changes to which factors determine if your A/B test results are significant.

To select a variation to send if your A/B test results are inconclusive, navigate to the A/B tab in your email editor > choose the preferred variation under If the results are inconclusive, send option.

User-added image

For more information on using A/B testing, check out this user guide article. To learn more about how to determine your test sample size and time frame in order to achieve statistical significance, take a look at this HubSpot blog post