10 Guidelines for Effective A/B Testing
- Run one test at a time - Testing more than one thing at a time muddles up the results. If you A/B test an email campaign that directs to a landing page at the same time that you’re A/B testing that landing page, your results can get muddled pretty easily. How would you know which change caused the increase in leads?
- Test one variable at a time - Same principle as above. In order to evaluate how effective an element on your page is, you need to isolate that variable in your A/B test. Test one element at a time.
- You can A/B test the entire element - While you certainly can (and should) test a different button color or a background shade, you should also consider making your entire landing page, call-to-action or email a variable. Instead of testing single design elements, such as headlines and images, design two completely different pages and test them against each other. Now you’re working on a higher level. This type of testing yields the biggest improvements, so consider starting with it before you continue your optimization with smaller tweaks.
- Test minor changes, too - Although it’s common to think that big, sweeping changes can increase your lead generation numbers, the small details are often just as important. While creating your tests, remember that even a simple change like switching the image on your landing page or the color of a CTA can drive big improvements. In fact, these sorts of changes are usually much easier to measure than the bigger ones.
- Measure as far down the funnel as possible - Sure, your A/B test might have a positive impact on your landing page conversion rate, but how about your sales numbers? A/B testing can have a significant effect on your bottom line. You may even see that a landing page that converted fewer prospects produced more sales. As you create your A/B test, consider how it affects metrics such as clickthrough rates, leads, traffic-to-lead conversion rates, and demo requests.
- Set up control & treatment - In any experiment, you need to keep a version of the original element you’re testing. When conducting A/B tests, set up your unaltered version as your “control” -- the landing page you would normally use. From there, build variations, or “treatments” -- the landing page you’ll test against your control. For example, if you are wondering whether including a testimonial on a landing page would make a difference, set up your control page with no testimonials. Then create your variation with a testimonial.
- Decide what you want to test - There are a number of variables you can decide to test. You don’t have to limit yourself to testing only images or text size. Look at the various elements on your marketing resources and their possible alternatives for design, wording, and layout. In fact, some of the areas you can test might not be instantly recognizable. For instance, you can test different target audiences, timing, alignment between an email and a landing page, and so on.
- Split your sample group randomly - In order to achieve conclusive results, you need to test with two or more audiences that are equal. With HubSpot, we automatically split traffic to your variations so that each variation reaches a random sampling of visitors.
- Test at the same time - Timing plays a significant role in your marketing campaign’s results, be it time of day, day of the week, or month of the year. If you were to run Test A during one month and Test B a month later, you wouldn’t know whether the changed response rate was a result of the different template or the different month. A/B testing requires you to run the two or more variations at the same time. Without simultaneous testing, you may be left second-guessing your results.
- Decide on necessary significance before testing - Before you launch your test, think about how significant your results should be in order for you to decide that the change should be made to your website or email campaign. Set the statistical significance goal for your winning variation before you start testing. Not sure what to shoot for? Try somewhere in the 97-99% range.
Follow these steps to create an A/B test with your landing pages.
1. Create a landing page
Go through the steps to create your first variation (the control), using the normal steps for building a landing page.
2. Create a new variation
Hover over the landing page you'd like to edit and you'll see the action buttons appear. Click Edit to continue.
Then click to the page's Settings to find the Create A/B Test link in the top right corner.
3. Choose a name for this variation
4. Edit the landing page variation to create a proper A/B test
Consider the following variables for testing:
- Offers - Experiment with the medium of the offer. You might test an ebook versus a whitepaper or video.
- Copy - Experiment with the formatting and style of the content. You could test plain paragraphs versus bullet points or a longer block of text vs a shorter block of text.
- Image - Try out different images to see how the conversion rate is influenced.
- Form Fields - Should your form only request an email address or should it ask for more information? Play around with the length of the form.
- Whole Page - The fastest way to achieve drastic results and produce a landing page that drives a lot of conversions is to test the entire page. Make variations on the whole page that affect image placement, form length, and copy. Once you have a statistically significant result pointing to a variation that performs better, you can continue optimizing through smaller tweaks.
You can quickly edit the other variation of the landing page using the edit variation A or B toggle button.
Want to delete the B variation before publishing your A/B Test? View this article to find out how.
5. View the A/B test results
- Use the Variations tab from the landing page details to view your A/B test results.
- Filter by several metrics such as Submissions, and Customers.
- You can also choose a variation as the winner of the test and stop serving up the other variation using the Choose as winner button.