A/B Testing

When to use it

Use this tool to create different versions of a marketing experiment to determine which digital campaign gets more engagement from your audience.

What’s the purpose

When marketing your idea, it is helpful to create different versions of your campaign to test with your audience. By changing single variables for the same design / content, you can determine which one performs better and is therefore more effective.

How to do it

Start with identifying variables to test. As part of the Testing category, in the first column, identify which variables you want to test. You need to create two versions of the same variable e.g. different calls to action for a specific user action.
Continue with formulating hypothesis. For each one, write down your hypothesis, in the second column, which is what you expect to find from conducting the experiment.
Now, as part of the Findings category, fill in the rest of the table using the prompts below, this will help you decide which tests to run first. Score them between 1 and 10.
- Impact: Impact looks at how much the test will make a difference for your user. For instance, will it make it easier to use or will it increase conversion?
- Confidence: Confidence is the certainty that the test will actually have the predicted impact.
- Ease: How easy is the test to run e.g. can you do it alone without a designer/ developer? 10 is very easy, 1 is very hard.
- Total: Total up your ICE scores for the specific test and review all your hypotheses: the ones with the highest scores should be prioritized first.
- Learning: After running your A|B test, what did you learn? Capture your learnings or key decisions here.

Helpful tips

You may identify more than one variable to optimize on a web page but test one at a time. These may be alternatives for design, wording and/or layout.

Using this method in your day-to-day work

When you have a marketing campaign that you plan to put out into the world, it can be challenging to land on the message and/or design that will capture the attention of your audience. The next time you are designing a marketing campaign, run an AB Test to see which of two different designs or two different messages is more effective. By simply changing one variable and seeing which version performs better, you can easily determine which one to put more effort and budget behind.