What is A/B Testing?

A/B testing is comparing two versions of a web page, button, hero image, newsletter or any component and compare which performs better. The decision is made using goals to compare against each other. The variation with better goal conversion wins.

The “Learn more” button is tested with two variations. Image Reference: link

Multivariate testing is the process of testing more than 2 variations of the component. This blog by Kissmetrics is an interesting post on A/B and Multivariate test – blog.

Conversion Metrics

The conversion can differ between websites based on the purpose of the website. Examples:

• Sign-Ups to newsletter for a publisher
• Purchase for e-commerce
• Outbound click to the target site for an affiliate
• Time on site for publisher
• Form submission for lead generators

Components to test

While the goals is very important to decide the winner, the component that will be tested is also equally important.

Facebook tested the message in social ads to make one “like” a page. Reference credits: link

Given the capabilities of tools in market, components like the following can be easily tested:

• Hero Images on home page or landing pages
• Call to action buttons
• Form elements (form length)
• Menu options
• Text in the titles, paragraph headings, the paragraph content etc.,
• Deals shown to users
• Newsletter email subject and content
• The price tiers and messaging for subscription or a product

This post is kept to be a simple intro to what an A/B testing is. This will be followed by posts on the A/B testing process, Implementing the test, Stats and factors involved in deciding the winner and few case studies from prominent A/B testing tools in the market.