A/B Testing Display Ads 101

You’ve mastered the basics of creating impactful display advertisements.  So, what’s next?  Well, as advertisers, we are continuously trying to strengthen and optimize not only our strategy, but the effectiveness of our creative units.  A/B testing allows us to examine single variables and compare their performance directly against each other, allowing the tester to tweak and optimize for the best marketing return.  

Google was at the forefront of adopting this process, which started about 12 years ago and used it to determine how many search results they should lay out on one page.  To give you an idea of how much they utilize A/B testing, in 2011, they ran over 7,000 tests.    

The beauty of A/B testing is we can use it for almost anything.  Take, for example, landing pages.  About 48% of marketers do not test varying landing pages, which means, their ability to optimize conversions on that webpage goes out the window.  So, whether it’s testing your landing page, banner ads, email subject line, or perhaps your shirt color, the technique works incredibly well for driving optimal engagement for every audience.  

Here is your 101 guide to advertising A/B testing:

1.  Keep It Simple

If your focus is ad creative, start as simple as possible.  Begin with the color scheme.  Test a palette of colors or single background and then a completely different color set on your next batch of ads.  Launch each campaign at the same time, with the same number of ad units, in the same sizes, with the exact same targeting and flight dates. By keeping all variables except one equal you will be able to isolate what is really driving performance. As a general practice, don’t try and test multiple variables at the same time as it will be challenging to determine which variable was the actual driver of change.

2.  Track Important KPIs

It’s critical to know what KPI you want to measure with your test. It’s best to have one main KPI that you are looking to improve then working towards it with methodical testing. In the example of ad creative, if your main KPI is your Click-Through-Rate (CTR), run your test with the goal of improving this one metric. Test different background colors or Call-to-Action language (eg “Shop Now” vs “Shop the Sale”) to see what moves the needle most.

3.  Test.  Optimize.  Repeat.

Once you complete your first A/B test you’ll have a baseline for how the two variables performed. For example, if you ran a test to see if a blue or red background increased engagement on ads and blue was the clear winner, your next step will be to iterate on your blue ad. Maybe on the next test you can try sky blue vs cobalt blue. From here, you can drill down even further. Once you have a clear winner on background color, use that winning color scheme and perform an A/B test on the call-to-action (CTA).  When you have your CTA winner, perform an A/B test on your headline copy, logo color, etc.  Small incremental changes can have a dramatic impact on your business and your bottom line.

4.  Manage Your Time

There are countless permutations you can choose from and plenty of things to A/B test, so be careful not to get carried away.  Once you’ve found something that works and drives the performance you desire, stick with that method for the time being and work on managing your new traffic.  After you’ve developed strategies around managing the influx in traffic, work on scaling up again using A/B testing as a tool.


In advertising, it’s easy to see how this trend started with the digital age and how it will keep expanding.  Now that ad space is cheap, and emails are free, what was once unfeasible financially for smaller companies is now attainable.

This practice even extends outside of the eCommerce industry to politics.  During his 2008 campaign, President Obama tested call-to-action buttons on his display advertisements.  What resulted was a 40.6% increase in sign-ups, $60million in funding, and 288,000 additional volunteers all from the “Learn More” call-to-action his campaign adopted.  

By using the outlined guide to advertising A/B testing, you’ll be on your way to marketing and advertising much more intelligently.