Getting to Know Technology: A/B Testing

Defining A/B Testing
An A/B test is basically comparing two options to find out which one is better or more efficient for your purposes. A/B testing is meant to be kept simple so that there are as few variables involved as possible. A/B tests are used to make choices based on the variables, such as finding out which call-to-action is most efficient or which page layout is most effective. Option A is the way things currently are, or the control, whereas Option B should include the change, or the variables. You then expose an audience to each option in the same capacity to determine which is more effective.

Setting Up an A/B Test
While A/B testing can be used to make many different kinds of decisions, these are best handled one at a time. If multiple variables are addressed in a single test, chances are that the test won?t be anywhere near specific enough to make any well-supported decisions. A/B testing is best used when it is addressing small changes rather than large ones. You first need to identify the variable you want to test, as well as a metric that you want to measure it against. How can the change improve engagement? Can it increase the time that a user spends on the page? Can it provide a higher click-through rate? Like we mentioned before, it?s all about what you?re trying to measure or improve.

Once you?ve determined this, you want to determine what the control option is, as well as what change you want to make after that. Your control group will be what you are currently using, so keep in mind that you?ll have to judge the results of the change against this to determine if there is a significant improvement. You also need to determine the sample size, or the number of recipients who will be a part of the test. Of course, not all changes will be accurately measured by just your sample size. You might find that the study benefits a constantly running project that can collect an adequate amount of data. Before making any changes to your testing, you?ll have to figure out what your threshold for making changes should be so you don?t pull the plug on a test prematurely.

Running an A/B Test
As far as actually administering the test, you should keep the following in mind; be sure to give it enough time to collect data for your conclusion, and make sure options are tested at the same time to keep other variables from mucking up the data.

What are some ways A/B testing can help your business? Let us know in the comments.