In this article we’re going to cover some best practices, or “rules” to help you get the best results when split testing. But, before we get into the rules, we need to define some terms so they’ll make sense to you later on.

Control Page — this is your default page that doesn’t have any changes and will be compared with other pages that do contain changes. Generally speaking, this is currently the best converting page (or only page) you have so far and you want to see if you can create a page that does even better.

Variation Page — this is your new unproven page that has certain elements changed, such as a headline or an image, so that it can be compared against the control page.

Statistical Significance — If we say that a test has “statistical significance”, that means that there is a 95% confidence-level that the result isn’t attributed to pure chance.

Now that you know some basic definitions, let’s get into the rules!


Rule 1: Test the Big Things First

There are many things that you could split test: the color of your button, the size of your headline, the word at the bottom of your page, and so on…

But these are very small changes that won’t make much of an impact to your overall conversion rates.

For starters, here are the six things that you want to split test first in no particular order:
 

1. The headline
2. Images
3. Copy text
4. Button copy and other call-to-action text
5. Social proof
6. Video
    

The real key here is that you want to test these six items in radically different ways, for example an entirely different headline, a completely different image, and so on.

Don’t just change a word or two in your headline – change the entire message and see what the results will be. The same idea applies to the other items on the list, such as your images and social proof.

Sure, changing the color of your button might increase your conversion rates by 0.5% after testing for a month …

But is it really worth spending a month split testing the button when you could have been testing the headline and seen a 50% increase in conversion rates instead?

The other issue with split testing small items is that it will require way more traffic and therefore take a lot more time to reach statistically significant results (results that aren’t attributed to pure chance).

As a result, some of these small changes never reach a high level of significance, rendering the entire test useless.

So just remember to test the big things first before moving onto the smaller items.


Rule 2: Test Only One Thing at a Time

Split testing is fun, we get it.

You want to test your headline, your images, and the text copy all at the same time… But if you test all these things at once and see an increase or decrease in conversion rates, you won’t know which factor caused that change…

So keep it simple and split test one thing at a time.

First test the headline and get the results. Then test the images and get the results, and so on.

The main takeaway here is to make sure that you test only ONE thing at a time.


Rule 3: Don’t Make Any Changes Until the Test Is Done

You might get really excited and end a test early if you see a variation is underperforming or outperforming the control page, but you should never end a test early despite what your gut feeling is.

Instead, wait until ClickMagick automatically determines that your results are statistically significant and declares a winner for you. Determining statistical significance is complicated and requires a lot of math which is why this is built into ClickMagick so that you never have to think about it.

Let’s take a really quick example. Let’s say you have two pages that you’ve just started split-testing, and after 10 minutes, you see that one has a whopping 100% conversion rate whereas the other page has a paltry 25% conversion rate. Wow, have you got a winner! You can stop testing now, right?

Except wait… Let’s look into this deeper. You’ve set up your control page to get 75% of the traffic and your new unproven page to get only 25% of the traffic. Now you look at your unproven page, and it’s gotten exactly 1 unique click and that click happened to convert, so it has a 100% conversion rate.

Looking at your control page, you find that it’s gotten 4 unique clicks and only one of those converted giving you a 25% conversion rate.

While it is absolutely true that one page has a 100% conversion rate and the other has only 25%, those results are not statistically significant in any way because there are so few clicks. The next 5 clicks that come in could dramatically change those results.

If two of those 5 clicks went to your unproven page and didn’t convert and 3 went to your control page and two of those 3 did convert, then then unproven page would have a 33% conversion rate and the control page would now have a 43% conversion rate, making it the new winner—for the moment.

The lesson here is to forget about short time periods and low numbers of clicks — either one can give wildly misleading results.

There’s much more to computing statistical significance than this simple example, but hopefully this gives you an idea of why you can’t just look at the numbers and trust your gut.

Depending on how much traffic you’re sending, determining a winner can take hours, days, weeks, or even months. Be patient and don’t make any changes until the results are in.


Rule 4: Test One Variation at at Time

The more variations you have, the longer it will take to get statistically valid results.

For example, if you’re split testing two pages and receive 1,000 clicks, that gives about 500 clicks per page, assuming a 50/50 split.

But if you’re testing 10 different pages at once, you’re now splitting the same 1,000 clicks between 10 pages, only getting about 100 clicks per page.

And 100 clicks doesn’t provide the same useful information that 500 clicks would.

The only reason you’d want to test more than one variation at one time is if the variations are closely related and tied to each other, such as if you have different pricing plans and want to test them against each other.

But for other items, such as ad copy or images, testing only one variation will almost always yield statistically significant results faster.

So make sure to test one variation at a time.


Rule 5: Always Be Testing

No matter how amazing you are at marketing, you should never stop testing.

All the top marketers in the world like Neil Patel, Russell Brunson, Todd Brown are constantly split testing their campaigns.

Even Google and Facebook, the richest companies in the world still split test their pages. You can see this yourself when you log into their ad platforms – the layout changes constantly.

So keep on testing because no matter how good you think you are, there are just some things that will always come to you as a surprise when you split test.

It might also be a good idea to follow some of the top marketers and pick up some split testing pointers from them.

For example, here is an excellent blog post from Neil Patel talking about split testing:

How to Master A/B Split Testing Quickly (And Increase Conversion Rate)


So that’s about it for our quick “rules”

Ready to start split testing yourself? Here’s a good blog post that covers other common mistakes when split testing and how to avoid them:
 

13 Dumb A/B Testing Mistakes That Are Wasting Your Time

To set up your own split test with ClickMagick, follow the simple steps in this article:

How do I set up a split test?

Article 468 Last updated: 12/07/2022 3:34:39 PM
https://www.clickmagick.com/kb/?article=468