The "Tech. Arch."

Green Behavior-Driven Development With Cucumber

DDT / Development-Driven Tests

For the last several years I have been generalizing the use of automated unit tests on my team (at work) as part of the development process for our C#-based enterprise services and business applications. Circa 2005, I was using NUnit and NAnt and Cruise Control to implement continuous integrations builds. Then as we implemented Team Foundation Server, we switched to MS Test. We created unit tests focused on specific classes as well as integration tests focused on interactions involving multiple classes. I have usually tried to strike the balance on not creating test for the sake of test but mostly to test complex logic and ensure key enterprise services would work reliably. Automated tests have been part of our coding process but not upfront as the Test-Driven Development practitioners would propose. I have been resistant to TDD for a couple reasons:

Early on, developers need to focus on understanding the target functionality
Our lifecycle includes some up-front time-boxed design focused on components and their interactions
It is easy to fall deep in the rabit hole of creating too many detailed tests with all their associated fixture
If the depth and breadth of tests are inconsistent the cost of the development phase can quickly go up

Our practice is really DDT i.e. Development-Driven Tests, as opposed to TDD (Test-Driven Development).

The Missing Ingredient In The Recipe

So an ingredient is missing: how to get developers to absorb the new functionality in palatable chunks while yet conjuring and capture the interesting scenarios that might influence the structure of the code.

Focus on Behavior And Acceptance Is Key

My outside-work interests in Ruby, Web Frameworks, and Agile approaches recently brought me to dig into the different types of testing frameworks associated with Rails projects such as:

RSpec – the father of Ruby-based BDD frameworks
Shoulda – alternative to RSpec trying to make the syntax a bit easier to read
Cucumber – set the tone by focusing the author on the “Given-When-Then” structure

There are probably other frameworks but so far I have only looked into these three.
These frameworks aim at raising the level of abstraction above the traditional “x-unit”-style unit tests which are typically focused on defining test methods containing specific code-level assertions such as:

TestUnit Example:

class MyTest < Test::Unit::TestCase
  def do_some_work
    # ...
    assert_equal("OK", result)
  end
end

Rspec Example:

require 'rubygems'
require 'spec'

describe "Skills Tracking" do
	it "should track skills for a given user" do
	end
end

Running the spec produces:

These frameworks are very convenient to build acceptance either ahead or during development but they still set the expectation that you are writing at least some minimal code. You will also notice that even though help create what is called "syntactic sugar" to help with readability, authoring a spec requires that you at least know quite a bit the syntax. What if the abstraction level could be raised one more level and guide you through the BDD process?

Adding Cucumber To The Recipe

Well, search no more! Cucumber help focus the author on the key tenet of BDD:

Define each business Feature - using the syntax:

In order to...
As a...
I want...

Feature: Skills Tracking
  In order to manage my skills
  As a person continuously developing skills  
  I want to be track my skills

Define multiple Scenarios where a feature can be exercised - using the syntax:

Scenario: Track New Skill
  Given a new skill 
  When I define a new skill
  Then the new skill will be tracked
 
Scenario: Update Skill Proficiency
  Given an existing skill 
  When I have increased my proficiency level with that skill
  Then the skill proficiency level will be updated

Notice that I have not used any code and very minimal syntax. The approach also leads you to define scenarios as you specify your feature. I don't need to think about code quite yet.

Tasting The Cucumber

So let's see what minimal infrastructure is needed. I am assuming you have Ruby. If not, check my post titled Getting Started With Ruby And Rails. To keep things to a minimum, I am going to show you how to create your features with only Ruby (as opposed to depending on the whole Rails stack).

Install the cucumber gem

gem install cucumber

Create a project folder (e.g. "Cuke")
Create a file in Cuke, name it rakefile and edit it to enter the following content:

require 'rake'
require 'rake/testtask'
require 'rake/rdoctask'

require 'cucumber/rake/task'
 
Cucumber::Rake::Task.new do |t|
  t.cucumber_opts = "--format pretty"
end

Create a file in Cuke, name it cucumber.yml and edit it to enter the following content:

autotest: -r features --format pretty --color

Create a lib folder below Cuke
Create a tasks folder below Cukelib
Create a file in Cukelibtasks, name it cucumber.rake and edit it to enter the following content:

begin
  require 'cucumber/rake/task'

  Cucumber::Rake::Task.new(:features) do |t|
    t.fork = true
    t.cucumber_opts = ['--format', (ENV['CUCUMBER_FORMAT'] || 'pretty')]
  end
  task :features => 'db:test:prepare'
rescue LoadError
  desc 'Cucumber rake task not available'
  task :features do
    abort 'Cucumber rake task is not available. Be sure to install cucumber as a gem or plugin'
  end
end

Create a features folder below Cuke
Create a file in Cuke, name it testcuke.feature and edit it to enter the following content:

Feature: Skills Tracking
  In order to manage my skills
  As a person continuously developing skills  
  I want to be track my skills

Open a command window on our Cuke folder and let's run Cucumber

cucumber features/testcuke.feature

The result is:

Let's add a scenario to the feature:

Scenario: Track New Skill
  Given a new skill 
  When I define a new skill
  Then the new skill will be tracked

Let' re-run Cucumber

cucumber features/testcuke.feature

The result is:

Cucumber identified the steps of the scenario and colored them amber to indicate that they are not implemented yet. So let's tackle one step.

Create a step_definitions folder below Cuke
Create a file in Cukestep_definitions, name it skill_step.rb and edit it to enter the following content based on the template suggested by Cucumber:

Given /^a new skill$/ do
  pending
end

Let' re-run Cucumber

cucumber features/testcuke.feature

The result is:

To make the test pass we would need to fully implement the step.

Incremental BDD

So far, everytime we have refined the feature, its scenarios and steps, we have re-run cucumber. What if Cucumber could run as soon as a change has been made (i.e. a file has been updated).
Enter autotest, a "file-watcher" which will trigger a new run of Cucumber.

Install the ZenTest gem

gem install cucumber

Set the following environment variables in your command line session

SET AUTOFEATURE=true
SET HOME=.

Run autotest

autotest

Now when you make a change to any file, you will see autotest run Cucumber.

So What?

Looking back at TDD (or DDT!) practices, it is clear that the focus of testing is more on code artifacts than on functional features.
Behavior-Driven Development can help a development team absorb and distill business requirement into acceptance test scenarios from which automated tests can be derived.
Even if you are not using Ruby or Rails as your development platform, Cucumber and Ruby can still play a role during the design phase. And you can, over time, introduce the use of Ruby to create integration test with non-Ruby components either through web services, databases or web clients.
If your core platform is .NET and you are considering building rich clients with Silverlight, you should also consider IronRuby. In that scenario, Cucumber can directly be leveraged with IronRuby.
Over time, I am envisioning the following for my team:

We start leveraging Cucumber to map some of the key functional requirements into features (but from a developer's standpoint)
When ready we derive high-level integration tests using IronRuby
We create C#-based unit tests
We then combine IronRuby integration tests with lower-level unit test-level harnesses

I am clearly planning to further develop my knowledge and experience with BDD through Cucumber.

References and Resources

Sites:

BDD - Behavior-Driven Development
Dan North first article on BDD
RSpec - the father of Ruby-based BDD frameworks
Shoulda - alternative to RSpec trying to make the syntax a bit easier to read
Cucumber - set the tone by focusing the author on the "Given-When-Then" structure
Outside-In Development with Cucumber: Ben Mabey