Knowledge Base      


How do I analyze my split test results in ClickMagick?



You can see the results of your split tests at any time using the Campaigns Funnel Steps report:
 


You’ll want to put the Steps Report into full-screen “comparison mode” by clicking here …
 


Then simply select the two paths that you want to compare. You can compare the results for any date range, and even for specific traffic sources, by modifying the filters at the top of the report as necessary.

 

 
Split testing with Campaigns isn’t yet fully automated, so you’ll need to manually analyze the results on your own.

When you can consider the results to be “accurate” will really depend on your business, but if we were you, we’d wait until we had about an average week’s worth of conversions before trying to analyze the results.

For example, if you normally generate an average of 100 sales a week, wait until at least one of the pages you’re split testing has at least 100 sales before analyzing your results and concluding the test.



To view the results of your split test, go back to the Tools menu and choose “Split Testing” again.

This will show you the stats for your control page and all of your split test variations, including the number of clicks, conversions and the conversion rates for each.

In addition to these basic stats for each page, ClickMagick will also show you how the conversions for each variation compare to your control page — represented by a percentage difference.

For example, if a variation has a Difference value of +78% highlighted in green, this means that particular variation is doing 78% better than your control page. On the other hand, a Difference of -42% highlighted in red means that your variation is doing 42% worse than your control page.

Even better, ClickMagick will also do all the complicated math to determine when you have a statistically valid winner.

If you tell ClickMagick that you want to optimize for Sales for example, once you have a statistically valid winner based on Sales ClickMagick will automatically end the Split Test and start routing all of your traffic to the winner in order to maximize the value of every click. You’ll also receive an alert via email.

If you do not want ClickMagick to do this, you can run what we call a Manual Test in which case ClickMagick will simply show you the results and you can end the test manually whenever you want.

Just keep in mind that until ClickMagick shows that you have a statistically valid winner or loser the results are not statistically valid and you should not make any changes based on these inconclusive results.

And just so you know, ClickMagick considers a result to be statistically valid when the “confidence interval” is 95% or higher, which is the generally accepted value to use for this purpose.
 
 
Note: Due to random chance, a page could theoretically get 10 clicks and 8 sales, even if your average conversion rate is only 3%. This type of thing is very rare, but to prevent this from ruining your split test ClickMagick waits until all your URLs have at least 100 clicks before attempting to determine a winner.
 
 
Note: If you want to stop a split test before a winner is chosen, simply delete all the split test URLs by clicking on the Delete icon next to each link.


Article 519 Last updated: 04/26/2023 11:13:57 AM
https://www.clickmagick.com/kb/?article=519