Abstract:

The key benefits of Behavior-Driven-Development (BDD) practices are communication enhancement and customer satisfaction. You can read more on that by Dan North and Gojko Adzic. Perhaps the biggest practical challenge that stands in the way of reaping those benefits is the burden of provisioning, installation and maintenance of requisite complex and fussy infrastructure. The recent availability of CI servers such as Jenkins & cloud-based testing services such as Sauce Labs  carries the potential to remove that barrier. This post discusses and shows how to integrate Behat, an emerging BDD framework for PHP, with Jenkins and Sauce Labs.

What is Behat?

Behat is a BDD framework for PHP. There are some tools available for BDD like Cucumber for Ruby, SpecFlow for .NET and Lettuce for Python. Behat is the first BDD tool for PHP applications. Developers can also use the  PHPSpec framework to implement classes within the Behat projects. Behat is written in PHP by Konstantin Kudryashov. With Behat, you can write human readable stories, which are used as tests to run against your application. Behat can be used for API testing, functional testing and data-driven testing. Developers will do API testing and we will carry on with functional testing (web acceptance testing) with Behat.

Functional Testing with Behat and Mink

Behat is used for acceptance testing (any tests) by executing a Gherkin scenario. Developers can implement integrated classes. Testers start thinking of more workflow level and technical level steps (actions), which turns scenarios to features. Once a tester starts to think of Web Acceptance Testing (functional testing) with browser interaction, another tool called “Mink” comes into the picture.

Mink is used for browser emulation (functional testing) where browser interactions takes place. As of now, these are the following Selenium drivers available for browser emulation.

• Selenium provides a bridge for Selenium RC  (Selenium 1).
• Webdriver provides a bridge for Selenium 2. (Facebook webdriver) for PHP. Currently Sauce Lab integration is not available with Webdriver.

Note: The Behat and Sauce Labs integration is currently only available for Selenium 1.

Behat In Action:

You must first have pear installed in order to proceed with Behat installation. Now we will run these commands from your terminal window:

$ sudo pear channel-discover pear.behat.org
$ sudo pear channel-discover pear.symfony.com
$ sudo pear install behat/gherkin-beta
$ sudo pear install behat/behat-beta

 

Test your installation by running this command:

$ behat --version
Behat version 2.2.0

 

Now let’s install Mink and run the following commands from the terminal window:

$ pear channel-discover pear.symfony.com
$ pear channel-discover pear.behat.org
$ pear install behat/mink

 

Mink is ready to use. We have to include “mink/autoload.php” in your “bootstrap.php” file as shown below:

require_once 'mink/autoload.php';

Start Your Project

Navigate to the project root directory and initialize Behat by running these commands:

$ cd /path/to/my/project

$ ls
application

$ behat --init
+d features - place your *.feature files here
+d features/bootstrap - place bootstrap scripts and static files here
+f features/bootstrap/FeatureContext.php - place your feature related code here

$ ls
application	features	

$cd features

$ ls
bootstrap

$cd bootstrap

$ls
FeatureContext.php

 

This will create a “features” directory and “bootstrap/FeatureContext.php” for you. Now we will jump directly to the project created with a feature file.

You can use NetBeans with installed Cucumber plugin for Gherkin syntax highlighting. Project structure will look like this:

Directory Structure

bootstrap.php

We can use this create file to define some constants and some third-party libraries that need to be included in class files. It’s not mandatory to have PHPUnit and SauceOnDemand extension installed unless you wish to make PHPUnit assertions in behat tests  Make sure you have installed correct version of PHPUnit which supports SauceOnDemandTestCase extension. You can follow SauceLabs blog to install it properly. This file should look like this:

<?php
date_default_timezone_set('Europe/London');
 require_once 'mink/autoload.php';
/*
 require_once 'PHPUnit/Autoload.php';
 require_once 'PHPUnit/Framework/Assert/Functions.php';
 require_once 'PHPUnit/Extensions/SeleniumTestCase.php';
 require_once 'PHPUnit/Extensions/SeleniumTestCase/SauceOnDemandTestCase.php';
*/

behat.yml

This file is a default config file that Behat uses to execute features. An example of behat.yml is shown below:

default:
    context:
        parameters:
            javascript_session: selenium
            base_url: http://en.wikipedia.org/wiki/Main_Page
            browser: firefox
            show_cmd: open %s

 

You can change the drivers by changing the “javascript_session” parameter. It can be “selenium” or “webdriver to run tests locally.

sauce.yml

This file is used for running features on Sauce Labs. The code for this file is explained in the “Behat and Sauce Labs” section below.

Note: Sauce Labs integration is available only with ‘selenium’ driver.

build.xml

This file is used for running features with ANT. We can use this ANT file to plug into Jenkins. Simple ANT file should look like this:

<project name="behat" basedir=".">
   <target name="behat">
    <exec dir="${basedir}" executable="behat" failonerror="true">
     <arg line="-f html --out ${basedir}/report/report.html"/>
    </exec>
   </target>
   <target name="runSauce">
     <delete dir="${basedir}/report" />
       <mkdir dir="${basedir}/report"/>
      <parallel>
      <antcall target="sauce"></antcall>
      </parallel>
   </target>
   <target name="sauce">
    <exec dir="${basedir}" executable="behat" failonerror="true">
     <arg line="-c sauce.yml -f html --out ${basedir}/report/report.html"/>
    </exec>
   </target>
</project>

There are two targets ‘behat’ and ‘runSauce’. The target ‘behat’ can be used for running tests locally and target ‘runSauce can be used to run tests on SauceLabs.

report

This directory is used to store HTML reports generated by Behat. The reports is stored in ‘report/report.html’ file.

Start your Engine

Remember, you have to download the latest version of Selenium server. Now navigate to the directory where you saved Selenium server .jar file. You have to launch it using command shown below:

java -jar selenium-server-standalone-2.15.0.jar

You need to start selenium server to run tests locally. In order to run tests on SauceLabs it’s not required.

Behat & Sauce Labs

Sauce Labs is a cloud testing service that allows Selenium tests to run in the cloud. Sauce Labs allocates machines and browsers for your tests, captures screenshots for every step and records video of all jobs (tests). You don’t need to set up separate machines to run tests. Sauce Labs helps us to write tests without complex infrastructure.

In order to integrate Behat with Sauce Labs, you need to have an account with Sauce Labs. You’ll need your “Username” and “API Key” to plug them into a config file.

Behat executes features with “behat.yml” file by default, but we can run features with any other configuration file. We can create another configuration file like “sauce.yml” to run features on Sauce Labs. Example “sauce.yml” should look like this:

default:
  context:
    parameters:
      default_session:    goutte
      javascript_session: selenium
      base_url:           http://en.wikipedia.org/wiki/Main_Page
      browser:            firefox
      selenium:
        host: ondemand.saucelabs.com
        port: 80
        browser: >
          {
            "username":         "your username",
            "access-key":       "your API key",
            "browser":          "firefox",
            "browser-version":  "7",
            "os":               "Windows 2003",
            "name":             "Testing Selenium with Behat"
          }

 

We will use “sauce.yml” as a config file to run features on Sauce Labs. If you wish to run all features from the “features” directory on Sauce labs, you can use this command:

behat -c sauce.yml

When to implement step definitions?

• If you can speak fluent Gherkin, then you don’t need to write code. Behat/Mink will understand Gherkin and run your features without suggesting step definitions.
• If the features are written by someone else, you can take full advantage of Mink’s APIs in order to implement step definitions suggested by Behat/Mink.
• It’s very important to write good Gherkin to write minimum code.
• It’s very easy to access Mink API’s by writing simple code.

Now we will see how you “click” particular elements on page. You need to use Xpath as a locator for that element. Mink will suggest some step definitions and you’ll need to complete it like this:

/**
* @Given /^I click Something$/
*/
public function iClickSomething()
{
$this->getMink()->getSession()->getDriver()->click("//Xpath");
}

 

Example  : Feature wikiSearch

We will write a simple feature to add a product into shopping cart. The feature will look like this:

 Feature: wikiSearch
  In order to search information on wiki
  As a Wiki user
  I want to get sensible results from site

 @javascript
  Scenario Outline: Search Keywords on Google
    Given I am on "/"
    And I fill in searchBox with "<input>"
    When I press search button
    Then I should see "<output>" 

    Examples:
      | input       | output         |                                   
      | London      | lʌndən/        |
      | NewYork     | nɪu ˈjɔək      |
      | Sydney      | sɪdni/         |
      | Mumbai      | मुंबई            |
      | Bejing      | 北京            |
      | Tokyo       | 東京            |
      | Lahore      | لاہور            |
      | Paris       | paʁi            |

 

Feature Explained

This feature file is written in “Gherkin” DSL (Domain Specific Language). The feature file mentioned above is a good example of data-driven testing. This feature will execute our scenario for 8 different data sets mentioned in the example section. This feature will have the following steps:

1. User enters “London” in search box.
2. User will check if that page has city name in their local language as described in output.
3. This test will run for 8 different cities as shown in “examples” section of feature.

This feature is also a good example of testing internationalization as it consists of test data (examples) in different languages.

Now we will run this feature using command:

behat --name wikiSearch

 

Remember, we are running it locally for now using default config file “behat.yml” with Selenium driver. After executing the above command, we will get some step definitions suggested by Behat/Mink

Feature: wikiSearch
  In order to search information on wiki
  As a Wiki user
  I want to get sensible results from site

  @javascript
  Scenario Outline: Search Keywords on Wiki # features/wikiSearch.feature:8
    Given I am on "/"                         # FeatureContext::visit()
    And I fill in searchBox with "<input>"
    When I press search button
    Then I should see "<output>"              # FeatureContext::assertPageContainsText()

    Examples:
      | input   | output    |
      | London  | lʌndən/   |
        Undefined step "I fill in searchBox with "London""
        Undefined step "I press search button"
      | NewYork | nɪu ˈjɔək |
        Undefined step "I fill in searchBox with "NewYork""
        Undefined step "I press search button"
      | Sydney  | sɪdni/    |
        Undefined step "I fill in searchBox with "Sydney""
        Undefined step "I press search button"
      | Mumbai  | मुंबई     |
        Undefined step "I fill in searchBox with "Mumbai""
        Undefined step "I press search button"
      | Bejing  | 北京        |
        Undefined step "I fill in searchBox with "Bejing""
        Undefined step "I press search button"
      | Tokyo   | 東京        |
        Undefined step "I fill in searchBox with "Tokyo""
        Undefined step "I press search button"
      | Lahore  | لاہور     |
        Undefined step "I fill in searchBox with "Lahore""
        Undefined step "I press search button"
      | Paris   | paʁi      |
        Undefined step "I fill in searchBox with "Paris""
        Undefined step "I press search button"

8 scenarios (8 undefined)
32 steps (8 passed, 8 skipped, 16 undefined)
0m15.771s

You can implement step definitions for undefined steps with these snippets:

    /**
     * @Given /^I fill in searchBox with "([^"]*)"$/
     */
    public function iFillInSearchboxWith($argument1)
    {
        throw new PendingException();
    }

    /**
     * @When /^I press search button$/
     */
    public function iPressSearchButton()
    {
        throw new PendingException();
    }

 

As you can see from above, Behat/Mink have suggested some step definitions for undefined steps. We can implement these step definitions using Mink in “bootstrap/FeatureContext.php” file. We can implement the first step definition “iFillInSearchboxWith($argument1)” like this:

/**
     * @Given /^I fill in searchBox with "([^"]*)"$/
     */
    /*
    public function iFillInSearchboxWith($input)
    {
        $this->fillField("searchInput",$input);
    }

 

Now, when we run “behat” command again:

behat --name wikiSearch

 

You can see now sixteen step passed. The terminal window output should be like this:

Feature: wikiSearch
  In order to search information on wiki
  As a Wiki user
  I want to get sensible results from site

  @javascript
  Scenario Outline: Search Keywords on wiki # features/wikiSearch.feature:7
    Given I am on "/"                       # FeatureContext::visit()
    And I fill in searchBox with "<input>"  # FeatureContext::iFillInSearchboxWith()
    When I press search button
    Then I should see "<output>"            # FeatureContext::assertPageContainsText()

    Examples:
      | input   | output    |
      | London  | lʌndən/   |
        Undefined step "I press search button"
      | NewYork | nɪu ˈjɔək |
        Undefined step "I press search button"
      | Sydney  | sɪdni/    |
        Undefined step "I press search button"
      | Mumbai  | मुंबई     |
        Undefined step "I press search button"
      | Bejing  | 北京        |
        Undefined step "I press search button"
      | Tokyo   | 東京        |
        Undefined step "I press search button"
      | Lahore  | لاہور     |
        Undefined step "I press search button"
      | Paris   | paʁi      |
        Undefined step "I press search button"

8 scenarios (8 undefined)
32 steps (16 passed, 8 skipped, 8 undefined)
0m25.568s

You can implement step definitions for undefined steps with these snippets:

    /**
     * @When /^I press search button$/
     */
    public function iPressSearchButton()
    {
        throw new PendingException();
    }

We have to repeat this process until all steps get “passed”. We can implement these steps by adding some code in “bootstrap/FeatureContext.php” file. The code will look like this:

<?php
use Behat\Behat\Context\ClosuredContextInterface,
    Behat\Behat\Context\TranslatedContextInterface,
    Behat\Behat\Context\BehatContext,
    Behat\Behat\Exception\PendingException;
use Behat\Gherkin\Node\PyStringNode,
    Behat\Gherkin\Node\TableNode;
use Behat\Mink\Behat\Context\MinkContext;
use Behat\Mink\Session;
use Behat\Mink\Driver\DriverInterface;
require_once 'bootstrap.php';

/**
 * Features context.
 */
class FeatureContext extends MinkContext
{
   /**
     * @Given /^I fill in searchBox with "([^"]*)"$/
     */
    public function iFillInSearchboxWith($input)
    {
        $this->fillField("searchInput",$input);
    }

    /**
     * @When /^I press search button$/
     */

    public function iPressSearchButton()
    {
        $this->getMink()->getSession()->getDriver()->click("//*[@id='searchButton']");
         $this->getMink()->getSession()->wait("3000");
    }

After implementation, we have to run “behat” command again. You will see the feature running in the browser for all the inputs mentioned in the feature file. You will have to wait until the browser finishes all the examples. You will see the terminal output look like this:

 Feature: wikiSearch
  In order to search information on wiki
  As a Wiki user
  I want to get sensible results from site

  @javascript
  Scenario Outline: Search Keywords on wiki # features/wikiSearch.feature:8
    Given I am on "/"                       # FeatureContext::visit()
    And I fill in searchBox with "<input>"  # FeatureContext::iFillInSearchboxWith()
    When I press search button              # FeatureContext::iPressSearchButton()
    Then I should see "<output>"            # FeatureContext::assertPageContainsText()

    Examples:
      | input   | output    |
      | London  | lʌndən/   |
      | NewYork | nɪu ˈjɔək |
      | Sydney  | sɪdni/    |
      | Mumbai  | मुंबई       |
      | Bejing  | 北京      |
      | Tokyo   | 東京       |
      | Lahore  | لاہور       |
      | Paris   | paʁi      |

8 scenarios (8 passed)
32 steps (32 passed)
0m42.794s

We managed to get all our scenario/steps “passed” so now it’s time to login into your Sauce Labs account to see this scenario running on Sauce Labs.

Now we have to use “sauce.yml” config file. We will run below mentioned command from terminal and we can see output on Sauce Labs as shown below:

 behat -c sauce.yml

Screenshots for the feature running on Sauce Labs:

You can see detail steps, screenshots and video of this job on Sauce Labs
See terminal output as shown below:

Building Features with Jenkins

Continuous Integration (CI) is one of the best practices in agile projects to detect bugs early. Each integration is verified by an automated build to detect integration errors as quickly as possible. Now we will see how we can integrate Jenkins to our behat project.

You don’t need to start selenium server to run features on SauceLabs.

The HTML reports generated by Behat can be plugged directly into Jenkins. Lets see how it can be achieved.

To continue, you need Jenkins installed on your machine. Now start Jenkins by executing *.war file from terminal on port 8080. You also need to visit “http://localhost:8080″ to see Jenkins GUI.

$  java -jar jenkins.war

• Visit http://localhost:8080/ and you should see Jenkins Dashboard.
• Create New Job for behat project called “BehatSauce”. Specify it as a ‘Build a free-style software project’.
• Specify your SCM. Here is Git repository
• Select ‘Invoke Ant’ from ‘Build’ and specify ‘runSauce’ target
• Specify HTML report path. (You need to have HTML report plugin installed on Jenkins)

• Save this settings and Click on “Build Now”
• Login to your SauceLabs account. You will see tests running there.
• Sit back and enjoy your features running on Jenkins till it finishes job.
• Few minutes later, you will see your build is “Green” and report generated as ‘Sauce Report

• Click on ‘Sauce Report’ and enjoy nice looking HTML report generated by Behat.

• You can watch full video of job executed on SauceLabs here

Clone Source Code from GitHub

You can try these steps on your own. Source code is available on Github.
Steps to follow:

• Clone the repository from my GitHub

       $ git clone git@github.com:Shashi-ibuildings/Behat-Sauce.git
       $ cd /path to/Behat-Sauce

• Edit ‘Sauce.yml’ file in order to specify your username and API key.
• Run ANT command to execute features on Sauce Labs

       $ ant runSauce

• Alternatively, you can configure your Jenkins job as mentioned earlier and watch your tests running on Sauce Labs.
• You can use this configuration in your project by changing urls in ‘behat.yml’ and ‘sauce.yml’ and writing your own features.

Conclusion:

We can write web acceptance tests with Behat and Mink combination. We can plug them into Sauce Labs with config file (“sauce.yml”) and run them on a CI server. Now you can implement BDD practices for PHP applications by utilizing the benefits of Bahat and Sauce Labs.

Demo

To see a video demonstration of this blog post, visit on Vimeo and YouTube.  To read a little more about me, click here.

This is part 2 in a 3-part guest blog post series by Jason Smiley, a QA Engineer at Gerson Lehrman Group.

In my last blog post, I showed what testing as a whole would look like, regardless of what we actually need to test. Just to recap, we have tests that check and validate results, actions that are a set of steps to take before having a value we can check, and pages that can be checked or interacted with.

For the sake of this article, let’s assume we will be testing several different websites with different code bases that are partially related. For example, we will have an admin site that can add, edit, and remove content from a different public content site. Assuming we have two different test projects, or one for each site, the first thing we need to do is create a base test, action, and page class that can be used by either test project to reduce the amount of code that needs to be written. These can also reduce any maintenance that needs to be done on updating common functionality.

Later, we can have another set of base classes that extend the shared base classes but are more specific to the project under test. Since there will be shared code, it is a good idea to create a separate project that can be referenced in your test projects. This allows you to change base code sets fairly easily, which can be useful if changing to a different testing framework.

Selenium Base Test Class

First, we’ll talk about a base test class. All tests need a browser, so first we must write a function that will create a browser. The tests will need to connect to the DB to allow for data checking, and it will also need utility functions so you can look at popups or switch between active frames. Let’s create SeleniumTestBase.cs class, which has these functions. In C#, it will look something like this:

public SeleniumTestBase
{
protected IWebDriver myDriver;

public SeleniumTestBase (IWebDriver webDriver)
{
myDriver = webDriver;
}

public DataTable GetData(string connString, string SQL)
{
//Some code which will query a DB for you
}

public IWebDriver StartBrowser(string browserType)
{
//Some code which will launch different types of browsers
}

public void SwitchToWindow(string handle)
{
//Some code which will switch windows for you
}

public void SwitchToFrame(string handle)
{
//Some code which will switch frames for you
}

public void LaunchUrl(string URL)
{
//Some code which will navigate directly to the specified URL.
}

public void Refresh()
{
//Some code to refresh the page you are currently on
}
}

Now, it’s important to note here that many of these functions don’t seem to be that hard to implement. In that case, why create a base class to encapsulate standard functionality already inherent to the Selenium Framework? There are two reasons for this.

One is that you will have less typing to do, which is always nice. The other is that, should you decide to change testing frameworks (from say Selenium 1 to Selenium 2), you will only need to update one class that handles all of the basic interactions. The ways of doing things may change from one framework release to the next. By encapsulating your code, you ensure that your code will always work the same as maintenance is performed since all tests will still call the same functions.

Selenium Action Base Class

Now we’ll move on to actions. Actions will execute the steps that will change the state of the website being tested (such as being signed in). Actions may also need to revert changes done by tests to a database for data cleanup, navigate to different pages, refresh the page, or interact with different frames and windows from the current active environment. Although the interactions between an action class and a page will be test specific, we can start writing the ways an action will interact with a browser or DB. A SeleniumActionBase.cs class will probably look something like this:

public SeleniumActionBase
{
protected IWebDriver myDriver;

public SeleniumActionBase(IWebDriver webDriver)
{
myDriver = webDriver;
}

public DataTable ExecuteSQL(string connString, string SQL = “”, string SPName = “”, string[] Parameters = new string[] {})
{
//Some code which will execute a query or stored proceedure for you
}

public void SwitchToWindow(string handle)
{
//Some code which will switch windows for you
}

public void SwitchToFrame(string handle)
{
//Some code which will switch frames for you
}

public void LaunchUrl(string URL)
{
//Some code which will navigate directly to the specified URL.
}

public void Refresh()
{
//Some code to refresh the page you are currently on
}
}

There are a couple of things I’d like to clarify about the above example when it comes to updating the database during test scripts:

• If you are going to manually execute SQL to clean up your work after a test, you should be able to do so by using custom SQL or by calling stored procedures directly (hence the optional parameters). In my experience, it is always better to use a stored procedure rather than using your own SQL to update a DB to ensure that you aren’t creating a data issue.
• You might not be able to actually update a DB directly using SQL due to internal security protocols. Or maybe you won’t even be comfortable doing this as your test code could break the DB. However, executing SQL to clean up your code is going to be much faster and more dependable than doing it through the UI. I’m not saying you should or should not clean up your tests this way, but if you are going to execute SQL to clean up your code, I’d advocate doing it this way.

Selenium Page Class

The last piece of the puzzle are the actual pages themselves. Pages are essentially just a grouping of page elements you wish to check and interact with. To code this the most efficient way, you should break the idea of a page into two separate classes.

One class, which I will refer to as a page class, is specific to the test project in the sense that it relates to the actual pages you want to test. From a coding perspective, these are the locator strings. Regardless of what framework you are using, you need to be able to find your elements. Typically, this is done by Id, XPath, or CSS, and generally won’t change unless the page you’re testing changes. Since some locator strings could be based on the browser start index, and browser start indexes can either be 1 or 0, the page class will need to know which browser is being used. Based on this, a SeleniumPageBase.cs class would look something like this:

public SeleniumPageBase
{
protected IWebDriver myWebDriver;

public SeleniumPageBase(IWebDriver webDriver)
{
myWebDriver = webDriver;
}

protected int browserIndex
{
get
{
return /*Code to get Browser index*/;
}
}
}

The second page class will handle the actual interactions between the test or action and the page. This is usually done by Selenium itself, however, a Selenium object isn’t always easy to use. A lot of times, you have to wait for the DOM to update the element you want to check, click, or interact with, especially if the page you are testing is using AJAX or Javascript. If an element fails, you might want to add additional error messaging to say what caused the interaction to fail. By using a level of encapsulation, you can better control the use of your Selenium or testing framework code.

Since this class is supposed to handle interactions of specific elements on a page, I will call this class a SeleniumPageElement. It is important to note that a page element class will need to mimic what a IWebElement –– or whatever framework you choose –– can do, as our actions and tests will be interacting with this class. By taking the testing framework’s place, we make it easier to swap out testing frameworks for new ones with little changes in test code.

Here is an example of what a SeleniumPageElement.cs class would look like:

public SeleneniumPageElement
{
public By myBy;
public IWebDriver myDriver;
public int? myIndex;
public SeleneniumPageElement myParent;

public SeleneniumPageElement(By locator, IWebDriver webDriver, SeleneniumPageElement parent = null, int? index = null)
{
myBy = locator;
myDriver = webDriver;
myIndex = index;
myParent = parent;
}

public IWebElement GetNow(string errorMessage = “”)
{
try
{
if(myParent != null)
{
if(myIndex != null)
{
return myParent.GetNow().FindElements(myBy)[MyIndex];
}
return myParent.GetNow().FindElement(myBy);
}

if(myIndex != null)
{
return myDriver.FindElements(myBy)[MyIndex];
}
return myDriver.FindElement(myBy);
}
catch(Exception e)
{
if(errorMessage.Equals(string.Empty) == false)
{
throw new Exception(errorMessage, e);
}
throw e;
}
}
public IWebElement WaitingGet(int seconds = 5, string errorMessage = “”)
{
//Waiting function using above method.
}

public void Click(int seconds = 5, string errorMessage = “”)
{
WaitingGet(seconds, errorMessage).Click();
}

In the last post of this series, I will go over how to connect what I covered in posts 1 and 2 and put it all together in your project.

Footnote: You will also need to write other waiting functions as well as interaction functions * to handle your framework calls. The above functions will handle the structure of the page element class to allow your other functions to handle specific interactions such as clicking or other types of waiting such as WaitUntilVisible(int seconds = 5, string errorMessage = “”) or WaitUntilNotVisible(int seconds = 5, string errorMessage = “”).

This is a guest blog post by Chris Johnson, a web developer, author and technology consultant. 

Cucumber and Selenium are a great way to bring browser testing to your Rails application. Selenium gives you the ability to automate tests that actually use the browser the way a human user might, giving you confidence that your application actually works. But one problem you may run in to is testing multiple browsers.

Sauce Labs comes to the rescue with the ability to let you easily run your tests in the cloud on multiple browsers.

Getting Started

Let’s hook up our rails application, Twitflickup, to work with Sauce Labs. To start, we’ll grab the application’s source code from github.

$ git clone git://github.com/johnsonch/twitflickup.git

With the repository cloned, we’ll want to run $ bundle install to get our basic dependencies installed.

This application requires a Flickr API key, which can be obtained from http://www.flickr.com/services/apps/create/. After getting an API key, we’ll want to copy env.yml.example and rename it env.yml, then replace the API key and secret from Flickr in this file.

Writing a Cucumber Test

Let’s start by adding a Cucumber test around the homepage of Twitflickup by creating the homepage.feature file inside of the features directory.

By opening up the app/models/mashup.rb file, we see that when a new mashup is created, the application searches Twitter for the word “fish,” then searches Flickr for the third word of the tweet.

For our homepage to look correct, we’ll want to make sure there is an image on it and a tweet with the word “fish” in it.

@selenium
Feature: Twitflickup searches twitter and flicker

  Scenario: Search twitter for fish and display tweet
    Given I am on the twitflickup homepage
    Then I should see a tweet with the word "fish" in it

  Scenario: Search flickr for an image
    Given I am on the twitflickup homepage
    Then I should seen an image from Flickr

With our feature file in place, let’s build out the step definition in features/step_definitions/homepage_steps.rb

Given /^I am on the twitflickup homepage$/ do
  page.visit("/")
end

Then /^I should see a tweet with the word "([^"]*)" in it$/ do |search|
  page.should have_content(search)
end

Then /^I should seen an image from Flickr$/ do
  page.has_selector?('#flickr-image img')
end

We now have our tests in place and can run them with the following command.

$ cucumber

After running the feature, we’ll get an output like the following:

Feature: Twitflickup searches twitter and flicker

Scenario: Search twitter for fish and display tweet    # features/homepage.feature:3
  Given I am on the twitflickup homepage               # features/step_definitions/homepage_steps.rb:1
    Then I should see a tweet with the word "fish" in it # features/step_definitions/homepage_steps.rb:5
      expected there to be content "fish" in "Twitflickup\n\nTwitflickup\n\n\n\n\nwho told you to come outside smelling like FISH and eggs ?\n\n@\nbaby_itsREALxX\n\n\n\n\n" (RSpec::Expectations::ExpectationNotMetError)
      ./features/step_definitions/homepage_steps.rb:6:in `/^I should see a tweet with the word "([^"]*)" in it$/'
      features/homepage.feature:5:in `Then I should see a tweet with the word "fish" in it'

Scenario: Search flickr for an image      # features/homepage.feature:7
    Given I am on the twitflickup homepage  # features/step_definitions/homepage_steps.rb:1
        Then I should seen an image from Flickr # features/step_definitions/homepage_steps.rb:9

        Failing Scenarios:
        cucumber features/homepage.feature:3 # Scenario: Search twitter for fish and display tweet

        2 scenarios (1 failed, 1 passed)
        4 steps (1 failed, 3 passed)
        0m10.009s

We are left with a failing test, but there is no way to see what’s going on. Sure, in this case, we can determine relatively easily that the failure is caused by us looking for the word “fish” and the tweet returning “FISH.” But wouldn’t it be nice to see a screen shot of what’s going on too? Luckily for us, we can easily set up our application to use Sauce Labs and take advantage of the ability to playback a test that’s been recorded to visually pinpoint what went wrong.

To run our Cucumber tests with Sauce Labs, we’ll need a Sauce Labs account and Sauce Labs API key. To get an account, sign up at https://saucelabs.com/signup and then grab the API key from the “My Account” page. For the purpose of this project, let’s copy the ondemand.yml.example file, rename it to ondemand.yml, and then replace the API and Username values with those we got from our own account. The ondemand.yml file will let Sauce Labs know who we are, which let’s us view our test runs from their web interface.

With our ondemand.yml file set up, we’ll need to include the Sauce gem in our gemfile. Inside of Gemfile, let’s add the following to the test group:

gem "sauce"

The Sauce gem is the Ruby client adapter that allows users to make Selenium tests run in Sauce OnDemand without any test changes. This gem can also be used with RSpec and Test::Unit, but here we’ll stick with Cucumber. The source for the Sauce gem is available at https://github.com/saucelabs/sauce_ruby.

Next we need to tell Cucumber about Sauce Labs, which will be set up in features/support/env.rb. We’ll need to require sauce, sauce/capybara, set up Sauce.config and change Capybara’s default driver to be sauce.

require 'sauce'
require 'sauce/capybara'

Sauce.config do |config|
  config.browser = "iexplore"

config.os = "Windows 2008"

  config.browser_version = "9"
end

Capybara.default_driver = :sauce

The Sauce.config section is where we can change what browser we want to test against. In this example, let’s stick with Internet Explorer on Windows 2008. Now that we have everything in place, we can run our Cucumber tests again. This time we’ll see our output letting us know we are connecting to Sauce OnDemand. By logging into Sauce Labs, we can see the job is executing and we can even click on the job name and watch the test execute in realtime.

Now that we have our tests executing on Sauce Labs, we can login to our account, see our test runs, and take advantage of Sauce’s great playback and screen shot features.

To find out more about using Cucumber and Selenium to test any website along with many other great web development tips, tricks and techniques, be sure to checkout Web Development Recipes from the Pragmatic Programmers.

Chris Johnson is a web developer, author and technology consultant living outside of Madison, Wisconsin. He has been developing websites professionally since 2003 when he got his first paycheck as a freelancer. When he’s not developing or writing, he enjoys tinkering with technology and mechanical things, photography, video games, playing hockey and spending time with his wife and their two dogs.

Twitter: @johnsonch
Website: http://www.johnsonch.com
Blog: http://blog.johnsonch.com