A/B Testing
What is Website A/B Testing?
A/B Testing Definition:
Website A/B testing is a method within marketing that evaluates the performance of two different versions of a website page, feature, or template in order to determine which version provides a better user experience and achieves better conversion rates. A/B testing helps website owners make data-driven decisions to optimize the design and functionality of their site, improve engagement and conversion, and enhance the overall user experience.
How Does A/B Testing Work?
A/B testing works by splitting up your website or landing page traffic into two groups (often a 50/50 split) and measures the results of your proposed design changes between the A (control) and B (variant) groups.
Website A/B testing can be deployed with the following steps:
Hypothesis development
Experiment Implementation
Analysis
Hypothesis Development:
A/B testing starts by developing a measurable hypothesis about your website or landing page and a feature you’d like to change. A hypothesis can follow the template of: By changing ____ into ____, we expect ____.
A/B Testing Hypothesis Examples:
By changing our primary CTA text from “Sign Up” to “Start for Free”, we expect to see an increase in click-through rate (CTR).
By reducing the number of navigation links on our landing page, we expect to see a reduction in bounce rate.
By introducing a mid-scroll CTA to our email sign-up within our blog, we expect to see an increase in our email sign-up conversion rate.
It is extremely important that when heading into an a/b test that there is a measurable result to help make a data-backed decision. We recommend setting one primary objective metric and a few secondary metrics as well that might help you inform your decision with the new design feature, messaging, etc. For example, a primary metric might be “conversion rate” while secondary metrics might be “avg. time on site” or “bounce rate”. If your conversion rate is the same between the control and the variant but there is a statistically significant improvement in behavior metrics like bounce rate or time on site, you might opt for the version that serves a better user experience.
Experiment Implementation:
After you’ve developed your hypothesis, it’s time to put your ideas to the test. The approach to testing your hypothesis will really depend on two main factors, which a/b testing tool you are using and how far the variant differs from the control design.
There are many A/B testing tools out there that you can use to help you setup and deploy split test experiments on your website with minimal or no-coding knowledge required! While there are many tools available, he most popular tools on the market for A/B testing include: Optimizely, VWO, AB Tasty, and Convert.
Once you’ve onboarded and installed your testing tool, it’s time to ask yourself: Can we simply manipulate the existing design through our tools visual editor (best for text/color changes or re-ordering of existing elements) or do we need to build an entirely new design? The difference here will determine whether you are going to do a standard “a/b test” or what is called a “redirect test”.
What is a redirect test?
A redirect test is where you build out an entirely new design on a separate URL and instead of the testing tool manipulating the web page on the fly through JavaScript, it instead redirects your defined % of users to a new URL. For example, your tool could send half of the users in your experiment to www.yoursite.com/landing-page/ and the other half to www.yoursite.com/landing-page-2/, measuring the same metrics across the non-redirected and redirected sessions.
Analysis:
After you’ve set up and launched your experiment, it’s important to spend ample time analyzing your results to make a data-backed decision about your new design. Most a/b testing tools will include measurement features that will inform you of the results of your hypothesis metrics between your control and variant. When looking at your modeled conversion rates across metrics, spend some time understanding if these results are “statistically significant” and what probability percentages the tool is assigning for each metric. It can also be good to double-click into dimension performance and view performance across device types, user categories, traffic source, etc. to garner even more insights. Finally, it’s recommended to keep track of your results. With
When Should I Use Website A/B Testing?
Ongoing A/B testing should be a part of any website owners Conversion Rate Optimization initiative, but is best leveraged for:
Testing new messaging and/or offers
Introducing new blocks or layouts to the website
Changing the website flow or layout
Altering the design on a high traffic area of the website
Changing the website’s primary call to action text or button design
Need Help with Your A/B Testing?
We can help. Surf CRO specializes in marketing analytics and conversion rate optimization services, including research, hypothesis development, and set-up of a/b tests. Let’s get to work!
