A/B Tests: The Secret Weapon For Great Marketing

Marketers don’t know anything. Any campaign that has ever run is based on an educated guess. Those involved in marketing use their experience and guesses to establish what they believe is going to be a successful campaign.

When you try and guess at something, you’re not always going to get it right the first time, or even the second or third time either.

Fake it till you make it?

In order to actually “make it”, you have to be really good at the guessing game. But you can make the guessing game work in your favor if you know how to use data from your campaigns to your advantage. That’s why A/B testing has become a necessity for any marketing program. A/B tests give you the ability to leverage the data you receive back on your campaigns in order to make them better, and better, and better.

What is an A/B Test?

An A/B Test is when you run an experiment for testing variations of something in order to see which variation yields the best result.

For a simplistic example, let’s say you’re running a Facebook Ads campaign to sell bracelets. In your Facebook Ad campaign, you can create multiple ads and run them at the same time.

Then as the ads are running, you’ll start seeing data on the performance of each ad. You can break down the number of clicks, the cost for each click, the number of conversions, the cost for each conversion, the return on ad spend, or whatever metrics you find appropriate for your campaign. By analyzing the data, you can establish which ad has the better performance and pivot towards the variations that perform better.

Benefits of Running an A/B Test

Learn what works

By running an A/B Test, you can learn about variables that help drive better performance. As you learn more about the things that work, you can do more of that. This helps you scale your campaigns with knowledge of things that work well for you.

Learn what works

Even more importantly, you can learn what does not work for you. Things that have low performance you know to stay away from. Don’t do those things again in future campaigns to avoid lower performance.

Increasing performance

As you remove poor-performing variations and enable more of the better-performing variations, you’ll make incremental performance lifts across the board for all of your marketing channels.

Continual improvement

Over time, these incremental performance lifts start to compound. By being consistent with your improvements over time, you’ll see major lifts in the performance of your marketing initiatives. You’ll dial in the best practices for your ads, landing pages, calls to action, and more. And you’ll be able to amplify that across all channels and continuously improve.

How To Run an A/B Test

Pick Something To Test

You can test all sorts of variables in an A/B test. Usually it’s best to test 1 variable at a time. If you change too many variables in your A/B test, you won’t know which change actually drove the change in results for your experiment.

Common Variables for A/B Test:

  • Ad Copy or Visuals
  • Web Page Layouts
  • Calls to Action
  • Promotional Offers
  • Email Subject Lines
  • Landing Pages

Identify Your Goal

Based on the variable you are testing, determine the KPIs that you want to improve with your test. You’ll want to monitor these KPIs before, during, and after your experiment. These are the metrics you’ll use to determine the winner of the A/B test.

  • Do you want to increase click-through rates?
  • Do you want to increase conversion rates?
  • Do you want to increase ROAS?
  • Decrease CPA?
  • Increase open rates?

Find the metrics that make sense for your test and take note of them.

Create a Variation

Have an “original” variation and then a new variation that you are testing against. Run them simultaneously side by side so you can see the results from the same time period. In your new variation, make one prominent change. That could be something visually like the image or video used, it could be a call to action button color, or maybe the entire layout in general.

Split Your Sample Group

Usually you’ll want to send 50% of the traffic the original variation and 50% the new variation. That way you have a good sample size for each version. If you’re testing something really extreme that is outside your norm, you could justify having your new variation being a smaller sample group.

Analyze Your A/B Test Results

Let the A/B Test run for a good amount of time. Enough time for you to collect a good amount of data for each sample group. It has to be a significant amount of data in order to be statistically representative. If you cut the test too short, or don’t push enough users into the test, you won’t be able to make a determination on the winner of the test.

As data comes in and you’ve collected enough data, dive into the KPIs you identified earlier. Which variations are driving the better results for you? Which ad is giving you more clicks? Which landing page is giving you the best conversion rate? Look at multiple stats to make sure you can declare a clear winner.

Depending on what you are testing, you’ll look at a variety of KPIs from your test:

  • Click-through rates
  • Cost per click
  • Conversion rates
  • Cost per conversion
  • Time on Site
  • Bounce Rate
  • Open Rates
  • Etc.

Make Your A/B Test Actionable

Most of the time, your data can show a clear winner. The variation that is driving the best performance is the version to implement as your final version. Turn off the losing variation and go “all-in” on the version that worked the best based on your KPIs.

Don’t stop there though. Just because that variation won that A/B test, doesn’t mean that’s the best possible option. It’s time to start your next A/B test to try something new. Start the process over again by picking a new variable to test. If you tested imagery before, this time test copy, or landing page, or call to action.

Keep running tests to constantly make things better and better. Don’t be stagnant because things can always improve. Plus, your winning variation will likely decrease in performance over time. User behavior changes and we want to continue to adapt in order to keep our marketing campaigns performing the best they can be.

A/b Testing Landing Pages

Here is an example of an A/B Test we did with an alternative landing page layout using Google Optimize.

In this test, we had an original landing page that we were pushing traffic to view paid ads. We were not sure if the landing page was the best it could be, so we created a second version of the page, changing the layout of information on the page.

We split the traffic between the two pages to see which one would convert better. Here is what we found.

You can see, the variation we created was converting traffic at a much higher rate. Instead of a ~2% conversion rate, the new page was getting us a 3.6% conversion rate. Significantly better.

Therefore, we replaced the original version of the page with our new updated version.

A/B Testings Facebook Audiences

Here is an example of an A/B Test we did running Facebook Ads.

In this test, we are trying to learn about what types of audiences respond best to our ads. Specifically, which audiences are going to drive the best conversions for an eCommerce store.

First, we created 3 different audiences into their own Ad Sets. We identified different groups of similar interests categories of the target audience for this store. This store sells plant-based products (food, plants, meals, etc.). Here is what we found.

You can see from the results that the “Plants” based audience performed significantly better than the other audience groups (Veganism and Organic Foods). Therefore, moving forward, we know to definitely tap into the “Plants” audience for future campaigns.

On the contrary, we learned that the “Organic Foods” audience did not seem to perform as well, so we should probably not target that group any longer since we know it doesn’t drive very great results.

Tools to Help With A/B Testing

There are a number of tools you can use to run your A/B test. Depending on what you are testing may change the tool you use to perform your testing.

A/B Testing Web Pages

This could be testing the copy of a page, the images on a page, the call-to-action on a page, or even testing the entire page layout. Google Optimize is a great choice. You can install this on your website for free and run unlimited page tests. With the chrome extension, you can make variations to test on the fly. Or you can split-test with a URL redirect to 2 different pages you already have built.

HotJar and Crazy Egg are other tools you can use to test your web pages. This is a heat mapping and website session recording software. You can use this to study how people interact with your pages. Are they scrolling to the right sections, are they clicking on the right buttons, are they finding the important information on the page, etc.

A/B Testing Email Marketing

This could be testing your subject lines, testing the layout, testing the preview text, or maybe testing the time of day you send an email. Many email marketing platforms have testing built-in, like MailChimp or ActiveCampaign. If you’re running an eCommerce business, you should definitely check out Klaviyo for your email marketing program and the robust testing functionality available there.

A/B Testing Ad Campaigns

Google Ads has “experiments” built right into the platform for any paid search and display campaigns you’d like to test. Facebook and Instagram Ads can also have A/B testing completed right in the ad platform.

Of course, if you need help running your A/B tests to maximize your marketing performance, we’re here for you.

Want help with your marketing?

Let’s connect and chat about your business goals.

More Valuable Reading

iOS 14 privacy updates are going to change everything about Facebook advertising. Here's how to get ready.
Use data to make your marketing campaigns better than ever.
The ultimate guide to building out robust Facebook Ads for the real estate market.

We bring brands to life online to maximize their digital footprint. Contact us today to learn more about working with us.