Over the past few months, while listening to TechZing, a fantastic podcast for web startup entrepreneurs, the concept of A/B Testing keeps coming up as key technique to track and measure conversion of site visitors into potential or true customers. Performing A/B testing implies:
- Selecting content variations or use case variations to test
- Identifying a couple alternatives for each test (a.k.a. experiments)
Example of an experiment on the wording of a call to action button:
- Trigger a test on a given page by selecting an alternative for a given user
- Track any resulting conversion for a given user on a targetted page
- Measure the participation in our tests
- Measure the conversion in our tests
A/B Test Solutions
Google provides very basic A/B Testing capabilities as part of the Google Analytics Website Optimizer. The downsides of that approach are:
- The tests (experiments), their alternatives, and corresponding urls have to be defined on the Google site
- All the data remains locked in Google’s databases (although you can export it)
If your web application is built on Ruby on Rails, you are in luck, as Patrick McKenzie has created ABingo, an excellent A/B testing framework plugin. ABingo even includes a dashboard to view the test participations and conversions.
Since I am a big fan of the lightweight Ruby Camping Micro-Framework, what I needed was an ABingo Camping plugin. Since none existed I emailed Patrick McKenzie to get his blessing and started to adapt his plugin!
Starting With The End In Mind
ABingo Test Application
To illustrate the usage of ABingo, we’ll use a simple tester representing a fictitious SAAS Application. From the home page we’ll be able to access the landing and corresponding sign-up pages that we want to experiment with.
We’ll also provide sign-in/sign-out options and a link to the ABingo dashboard if the signed in user is the administrator.
Objective: Increasing Our Sign-Ups
To increase our sign-ups we will be experimenting first with different calls to action in our sign-up button.
Our experiment named call_to_action will randomly vary the text of the button. We track the selected option and track the conversion on our sign-up page. After running our experiment for a while we will be able to see on the dashboard which option generated the most conversion!
The Conversion Funnel
The path taken by a group of customers from the various options (A/B/C) of the landing page to our sign-up page is called a conversion funnel:
High-Level Implementation Synopsis
Once ABingo is integrated in your application, tracking and measuring the conversion funnel will consist of 2 basic steps:
Selecting an alternative for a given experiment
In the landing view code we will invoke the ABingo ab_test API to choose an alternative for the call_to_action experiment based on an array of text options:
def landing # ... signup_text = ab_test("call_to_action", [ "Sign-Up Now!", "Try It For Free Now!", "Start Your Free Trial Now!", ]) a :href=>"/signup" do div.signup_btn! signup_text end # ... end
It’s that easy! And as a side note an experiment alternative can be a simple boolean, an array of arbitrary values, or even a hash of options.
Tracking the conversion for our experiment
In the SignUp controller, before rendering the signup view we will invoke the abingo! API to indicate that a conversion just occurred on our call_to_action experiment:
class SignUp < R '/signup' def get bingo! "call_to_action" render :signup end end
Fun With Experiments!
Now that you have seen how ridiculously simple it is to setup and track an experiment you'll want to try to run different types of experiments such as for example:
- different styles (size, color) for the sign-up button
- special offers
- different ways to present pricing options
Example of a landing page with a special_promo experiment:
Reviewing A/B Test Results With The ABingo Dashboard
The dashboard consists of a couple routes built using Camping controllers and views. Although we will walk through the setup of such a dashboard in our prototype app in a later article, here is a teaser screenshot:
The conclusions we can draw from these experiments are:
- For the special_promo experiment, the "true" alternative results in the most conversions (%)
- For the call_to_action experiment, the "Start Your Free Trial Now!" alternative results in the most conversions (%)
You can try the demo app yourself at: http://camping-abingo.heroku.com/.
- A/B testing is a key practice to improve usability as well as user conversion on a web application.
- Patrick McKenzie's ABingo is a powerful A/B testing framework for Ruby apps.
- ABingo makes it easy to define and execute experiments with multiple alternatives.
This introduction to ABingo should have given you a feel for how easy it is to add A/B testing capabilities to a Ruby Camping web application using the ABingo Camping plugin.
In a subsequent post I will cover the specific implementation steps to add ABingo to our test application. Stay tuned!
References and Resources
- A/B Testing Definition on Wikipedia
- Effective A/B Testing Basics (by Ben Tilly)
- Google Website Optmizer
- ABingo A/B Testing Framework (by Patrick McKenzie)
- Patrick McKenzie's Blog
- Patrick McKenzie Interview On TechZing Podcast