RSpec Story Runner Driven (Browser) Acceptance Testing

I’ve been using rspec now for quite a while now (thanks to chrissturm) and have been loving it. It feels a lot more natural and intuitive and I’m even getting the hand of learning when/how to mock/stub (though I still have some fixtures lying around). I’ve been meaning to learn up on using the new story runner feature and while googling I came upon a post by Kerry Buckley in which he provides a quick overview of how to setup story runner and also describes how he got story runner to drive selenium acceptance testing.

Just a quick blurb about what story runner is.

Story runner basically allows you to write specifications in a plain text file, written in natural language. You basically write a story (paraphrasing Dan North “a description of a requirement and its business benefit, and a set of criteria by which we all agree that it is done”.)

For each story you can then write a number of different scenarios (imagine that feature in different situations) and for each scenario you write a set of criteria which determines how that scenario can be completed successfully.

e.g. 
1
2
3
4
5
6
7
8
9
10
11
12
13


 Story: UI
  As a developer                                                  #
  I want to go to the uimockups page                              # Description of intent
  So that I can implement the mockup                              #

   Scenario: Going to the /uimockups page when not logged in      <= Scenario Description
     Given an anonymous user                                      #
     When the user goes to /uimockups                             #
     Then the document title should be 'personal'                 # criteria, actions & expectations
     And the page should contain the text 'done by Webtypes'      #  
     And the page should have a field named 'strip-search-input'  #
     And the page should have a form named 'strip-search'         #

Given a text file like this, you then write a small ruby script (see /stories/stories/project.rb below) which then takes the text, parses it look for the highlighted keywords. Each Given, When and Then is a Step. The Ands are each the same kind as the previous Step.

Run as is, you’ll get this same story output back to you but each of the lines under Scenario will be marked with “pending” which basically means that the story is yet to be implemented. e.g.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


saimon@artemis~/dev/projects/myrailsapp$ ruby stories/stories/project.rb
(in /Users/saimon/dev/projects/myrailsapp)
Running 1 scenarios

Story: UI

  As a developer
  I want to go to the ui page
  So that I can see the mockup

  Scenario: Going to the /ui page when not logged in

    Given an anonymous user (PENDING)

    When the user goes to /ui (PENDING)

    Then the document title should be 'personal' (PENDING)
    And the page should contain the text 'done by Webtypes' (PENDING)
    And the page should have a field named 'strip-search-input' (PENDING)
    And the page should have a form named 'strip-search' (PENDING)

1 scenarios: 0 succeeded, 0 failed, 1 pending

Pending Steps:
1) UI (Going to the /ui page when not logged in): Unimplemented step: an anonymous user

To actually get the story to pass, you need to implement each of the Steps in ruby. i.e. Here’s an example of the implementation of the 2nd step:

  • “When the user goes to /ui (PENDING)”
1
2
3
4
5
6


steps_for(:project) do
  When "the user goes to $path" do |path|
    get path
  end
end

As you can see it’s basically parsing the line for a step keyword (in this case ‘When’), and then takes the rest of the line and tries to match it against any of the When steps it knows about. It also goes one step further and allows you to add in variables so that you can extract dynamic criteria directly from the story line ($path) in this case.

So once, it has been matched, it ends up executing :

1
2


get /uimockups

The cool thing about it is that once you’ve implemented a step, it’s just reused every time it’s matched in the story. You can also have a stable set of steps which you use in multiple stories. You could even conceivably build up a library of them to be used in other applications.

I was at the BCN Ruby/Rails group meeting last night and one of the attendants expressed concerns about the brittleness of the syntax. In fact, there’s no problem because story runner will mark any line that it hasn’t been able to match against any of the steps known to it as pending so you can easily determine a syntax problem. And if an exception is raised by anything it has matched then it’ll provide the appropriate stack trace pointing you to the step that caused the exception.

After watching Pat Maddox’s screen-cast I’m convinced that using story runner is a good way of starting out your speccing. You can start by writing a story that describes a feature and then drill into it as you implement the steps. Along the way you’ll find you need to implement controllers, models, helpers and views and before you do you can then implement the appropriate specs (only enough to get the functionality in the story passing) which in turn drives the implementation of the object in question.

Now, finally, I can get to the real reason I wrote this post.

I’m interested in being able to do my integration tests via story runner and occasionally do the odd browser acceptance testing and as I also had been meaning to play with FireWatir & SafariWatir I decided to adapt his code to using watir.

But I added another requirement to the mix. What I really wanted was seamless integration between normal integration tests using plain rspec and browser acceptance testing hen running the same scenarios or even be able to mix and match.

After a bit I come up with this setup (very similar to Kerry’s original setup):

Directory Structure:

1
2
3
4
5
6
7
8
9
10
11
12
13


+-- lib
| +-- tasks/
|   +-- acceptance.rake
+-- stories/
| +-- all.rb
| +-- helper.rb
| +-- steps/
| | +-- project.rb
| | +-- watir.rb
| +-- stories/
| | +-- project.rb
| | +-- project.txt

Note: As per Kerry’s article I’ve further subdivided the top-level stories directory into stories and steps subdirectories. You don’t have to if you don’t have that many stories to write but I like the organized feeling it provides.

/stories/all.rb 
1
2
3
4
5
6


dir = File.dirname(__FILE__)
require "#{dir}/helper"
Dir[File.expand_path("#{dir}/stories/**/*.rb")].uniq.each do |file|
  require file
end

/stories/stories/project.txt

1
2
3
4
5
6
7
8
9
10
11
12


Story: UI
  As a developer
  I want to go to the ui page
  So that I can see the mockup

  Scenario: Going to the /ui page when not logged in
    When the user goes to /ui
    Then the document title should be 'personal'
    And the page should contain the text 'done by Webtypes'
    And the page should have a field named 'strip-search-input'
    And the page should have a form named 'strip-search'

/stories/stories/project.rb

1
2
3
4


#Call me with: [BROWSER=firefox|safari|ie] ruby stories/stories/project.rb
require File.join(File.dirname(__FILE__), "../helper") 
run_story_with_steps_for (browser ? [:watir_project, :project] : [:project])

/stories/steps/project.rb

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32


steps_for(:project) do
  Given "a test user" do
    User.delete_all
    User.create!(:name => 'test', :openid_url => 'http://dummy.openid/',
                :email => 'test@example.com')
  end
  
  When "the user goes to $path" do |path|
    get path
  end
  
  Then "the document title should be '$title'" do |title|
    response.should have_tag('title', title)
  end

  Then "the page should contain the text '$text'" do |text|
    response.should have_text(/#{text}/)
  end
  
  Then "the page should have a field named '$field'" do |field|
    response.should have_tag("input[type=text][id=?]", field)
  end
  
  Then "the page should have a form named '$form'" do |form|
    response.should have_tag("form[id=?]", form)
  end  

  Then "the page should have a submit button named '$name', with the label '$label'" do |name, label|
    response.should have_tag("input[type=submit][id=?][value=?]", name,label)
  end  
end

/stories/steps/watir_project.rb

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35


steps_for(:watir_project) do
  When "the user goes to $path" do |path|
    browser.goto "http://localhost#{path}"
  end
  
  When "the user types '$text' into the $field field" do |text, field|
    browser.text_field(:name,field).set(text)
  end
  
  When "the user clicks the $button button" do |button|
    browser.button(:value, button).click
  end

  Then "the document title should be '$title'" do |title|
    browser.title.should == title
  end

  Then "the page should contain the text '$text'" do |text|
    browser.text.include?(text).should be_true
  end
  
  Then "the page should have a field named '$field'" do |field|
    (browser.text_field(:name, field).exists? || browser.text_field(:id, field).exists?).should be_true
  end
  
  Then "the page should have a form named '$form'" do |form|
    (browser.form(:name, form).exists? || browser.form(:id, form).exists?).should be_true
  end  

  Then "the page should have a submit button named '$name', with the label '$label'" do |name, label|
    tf = (browser.text_field(:name, field) || browser.text_field(:id, field)).exists?().should be_true
    tf.value.should == label
  end
end

/stories/helper.rb :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60


ENV["RAILS_ENV"] = "test"
require File.expand_path(File.dirname(__FILE__) + "/../config/environment")
require 'spec/rails/story_adapter'

# watir gem
require 'firewatir' if ENV['BROWSER'] && ENV['BROWSER'] == 'firefox'
require 'safariwatir' if ENV['BROWSER'] && ENV['BROWSER'] == 'safari'

def start_ff
  FireWatir::Firefox.new
end

def start_safari
  safari = Watir::Safari.new
end


#Require steps in steps dir
Dir[File.dirname(__FILE__) + "/steps/*.rb"].uniq.each { |file| require file }

#Require appropriate watir browser object
if !$ff && ENV['BROWSER'] == 'firefox'
  $ff = start_ff_with_logger
end

if !$sf && ENV['BROWSER'] == 'safari'
  $sf = start_safari_with_logger
end

#Choose which browser to use in steps
def browser
  $ff || $sf
end

def run_story_with_steps_for *steps
  with_steps_for *(steps.flatten) do
    # Pull the filename of the caller out of the stack. Must be a better way.
    run caller[3].sub(/\.rb:.*/, '.txt'), :type => RailsStory
  end
end

# By default, RSpec adds an ActiveRecordSafetyListener to the story runner. 
# This rolls back database changes between scenarios, which is great if your calling your code directly, 
# but obviously means that if you write to the database, the server that Selenium's talking to can't see them. There's probably a cleaner way of disabling it.
class Spec::Story::Runner::ScenarioRunner
  def initialize
    @listeners = []
  end
end

module ::ActionController #:nodoc:
  module TestProcess
    # Work around Rails ticket http://dev.rubyonrails.org/ticket/1937
    # Helps to remove annoying html parser warnings
    def html_document
      @html_document ||= HTML::Document.new(@response.body, true, true)
    end
  end
end

So once you’ve got all that setup, you can then run:

1
2
3


saimon@artemis~/dev/projects/myrailsapp$ 
ruby stories/stories/project.rb

to execute the project story using basic rspec. It provides the following output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22


Running 1 scenarios

Story: UI

  As a developer
  I want to go to the ui page
  So that I can see the mockup

  Scenario: Going to the /ui page when not logged in

    Given an anonymous user

    When the user goes to /ui

    Then the document title should be 'personal'
    And the page should contain the text 'done by Webtypes'
    And the page should have a field named 'strip-search-input'
    And the page should have a form named 'strip-search'

1 scenarios: 1 succeeded, 0 failed, 0 pending

Woot! You can know just take that project.txt and send it to a client, a fellow developer, a project mailing list etc…

But, let’s go the extra step and run that same scenario against Firefox (go to FireWatir and follow the instructions. They’re pretty simple.)

Start firefox with -jssh 
1
2
3
4
5
6
7
8


saimon@artemis~/dev/projects/myrailsapp$ 
/Applications/Firefox.app/Contents/MacOS/firefox -jssh

run the story with the BROWSER environment variable:

saimon@artemis~/dev/projects/myrailsapp$ 
BROWSER=firefox ruby stories/stories/project.rb

and watch how FF is magically commanded to go through your stories scenarios. In the end it’s run the story against FF and provides the same passing output as the previous run.

One further step is to write a few rake commands to simplify running all your stories, with or without browser acceptance testing.

Add this file:

/lib/tasks/acceptance.rake

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47


desc "Run the acceptance tests, starting/stopping the test server."
task :acceptance_with_browser => ['acceptance:server:start'] do
  begin
    Rake::Task['acceptance:run'].invoke
  ensure
    Rake::Task['acceptance:server:stop'].invoke
  end
end
%w(firefox safari).each do |browser|
  Object.class_eval <<-EOS
    desc "Run the acceptance tests using the #{browser} browser."
    task :acceptance_with_#{browser} do
      $browser = '#{browser}'
      Rake::Task['acceptance_with_browser'].invoke
    end
  EOS
end
 
namespace :acceptance do
  desc "Run the acceptance tests."
  task :run do
    system "#{$browser ? "BROWSER='#{$browser}' " : ''}ruby stories/all.rb"
  end

  namespace :server do
    desc "Start the mongrel server"
    task :start do
      system 'script/server -e test -d'
      sleep 5
    end

    desc "Stop the mongrel server"
    task :stop do
      if File.exist? MONGREL_SERVER_PID_FILE
        pid = File.read(MONGREL_SERVER_PID_FILE).to_i
        Process.kill 'TERM', pid
        FileUtils.rm MONGREL_SERVER_PID_FILE
      else
        puts "#{MONGREL_SERVER_PID_FILE} not found"
      end
    end
  end
end

MONGREL_SERVER_PID_FILE = 'tmp/pids/mongrel.pid'

Then you can do:

1
2
3
4
5
6
7
8
9
10
11


saimon@artemis~/dev/projects/myrailsapp$ 
rake acceptance:run

or 

rake acceptance_with_firefox

or 

rake acceptance_with_safari

I thoroughly enjoyed getting that setup and though I only plan on writing browser specs for specific issues/features it’s nice to have the choice and the geek factor is way up high :)

Have fun speccing…

Torna a articles