Introduction to A/B Testing

What is A/B Testing?

A/B testing is comparing two versions of a web page, button, hero image, newsletter or any component and compare which performs better. The decision is made using goals to compare against each other. The variation with better goal conversion wins.

The “Learn more” button is tested with two variations. Image Reference: link

Multivariate testing is the process of testing more than 2 variations of the component. This blog by Kissmetrics is an interesting post on A/B and Multivariate test – blog.

Conversion Metrics

The conversion can differ between websites based on the purpose of the website. Examples:

  • Sign-Ups to newsletter for a publisher
  • Purchase for e-commerce
  • Outbound click to the target site for an affiliate
  • Time on site for publisher
  • Form submission for lead generators

Components to test

While the goals is very important to decide the winner, the component that will be tested is also equally important.


Facebook tested the message in social ads to make one “like” a page. Reference credits: link

Given the capabilities of tools in market, components like the following can be easily tested:

  • Hero Images on home page or landing pages
  • Call to action buttons
  • Form elements (form length)
  • Menu options
  • Text in the titles, paragraph headings, the paragraph content etc.,
  • Deals shown to users
  • Newsletter email subject and content
  • The price tiers and messaging for subscription or a product

This post is kept to be a simple intro to what an A/B testing is. This will be followed by posts on the A/B testing process, Implementing the test, Stats and factors involved in deciding the winner and few case studies from prominent A/B testing tools in the market.

One thought on “Introduction to A/B Testing

  1. Pingback: The A/B Testing Culture and Process | Tada – Data

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s