The "Tech. Arch."

A/B Test Your Ruby Camping Web App Using ABingo (Part 1)

Intro

Over the past few months, while listening to TechZing, a fantastic podcast for web startup entrepreneurs, the concept of A/B Testing keeps coming up as key technique to track and measure conversion of site visitors into potential or true customers. Performing A/B testing implies:

Selecting content variations or use case variations to test
Identifying a couple alternatives for each test (a.k.a. experiments)
Example of an experiment on the wording of a call to action button:
Trigger a test on a given page by selecting an alternative for a given user
Track any resulting conversion for a given user on a targetted page
Measure the participation in our tests
Measure the conversion in our tests

Click here if you want to skip straight to the ABingo Camping Plugin source code

A/B Test Solutions

Google provides very basic A/B Testing capabilities as part of the Google Analytics Website Optimizer. The downsides of that approach are:

The tests (experiments), their alternatives, and corresponding urls have to be defined on the Google site
The resulting “test tracking Javascript snippets” have to be copy/pasted inside your own site
All the data remains locked in Google’s databases (although you can export it)

If your web application is built on Ruby on Rails, you are in luck, as Patrick McKenzie has created ABingo, an excellent A/B testing framework plugin. ABingo even includes a dashboard to view the test participations and conversions.

Since I am a big fan of the lightweight Ruby Camping Micro-Framework, what I needed was an ABingo Camping plugin. Since none existed I emailed Patrick McKenzie to get his blessing and started to adapt his plugin!

Starting With The End In Mind

ABingo Test Application

To illustrate the usage of ABingo, we’ll use a simple tester representing a fictitious SAAS Application. From the home page we’ll be able to access the landing and corresponding sign-up pages that we want to experiment with.

We’ll also provide sign-in/sign-out options and a link to the ABingo dashboard if the signed in user is the administrator.

Objective: Increasing Our Sign-Ups

To increase our sign-ups we will be experimenting first with different calls to action in our sign-up button.
Our experiment named call_to_action will randomly vary the text of the button. We track the selected option and track the conversion on our sign-up page. After running our experiment for a while we will be able to see on the dashboard which option generated the most conversion!

The Conversion Funnel

The path taken by a group of customers from the various options (A/B/C) of the landing page to our sign-up page is called a conversion funnel:

High-Level Implementation Synopsis

Once ABingo is integrated in your application, tracking and measuring the conversion funnel will consist of 2 basic steps:

Selecting an alternative for a given experiment

In the landing view code we will invoke the ABingo ab_test API to choose an alternative for the call_to_action experiment based on an array of text options:

	def landing
		# ...

		signup_text = ab_test("call_to_action", 
								[ 	"Sign-Up Now!",
									"Try It For Free Now!",
									"Start Your Free Trial Now!",
								])
		
		a :href=>"/signup" do
			div.signup_btn!  signup_text
		end
		
		# ...
	end

It’s that easy! And as a side note an experiment alternative can be a simple boolean, an array of arbitrary values, or even a hash of options.

Tracking the conversion for our experiment

In the SignUp controller, before rendering the signup view we will invoke the abingo! API to indicate that a conversion just occurred on our call_to_action experiment:

	class SignUp < R '/signup'
		def get
			bingo! "call_to_action"
			
			render :signup
		end
	end

Fun With Experiments!

Now that you have seen how ridiculously simple it is to setup and track an experiment you'll want to try to run different types of experiments such as for example:

different styles (size, color) for the sign-up button
special offers
different ways to present pricing options
etc.

Example of a landing page with a special_promo experiment:

Reviewing A/B Test Results With The ABingo Dashboard

The dashboard consists of a couple routes built using Camping controllers and views. Although we will walk through the setup of such a dashboard in our prototype app in a later article, here is a teaser screenshot:

The conclusions we can draw from these experiments are:

For the special_promo experiment, the "true" alternative results in the most conversions (%)
For the call_to_action experiment, the "Start Your Free Trial Now!" alternative results in the most conversions (%)

You can try the demo app yourself at: http://camping-abingo.heroku.com/.

So What?

A/B testing is a key practice to improve usability as well as user conversion on a web application.
Patrick McKenzie's ABingo is a powerful A/B testing framework for Ruby apps.
ABingo makes it easy to define and execute experiments with multiple alternatives.

This introduction to ABingo should have given you a feel for how easy it is to add A/B testing capabilities to a Ruby Camping web application using the ABingo Camping plugin.
In a subsequent post I will cover the specific implementation steps to add ABingo to our test application. Stay tuned!