Adding Sauce To Behat

Posted by Shashikant Jagtap in Selenium Resources

Abstract: The key benefits of Behavior-Driven-Development (BDD) practices are communication enhancement and customer satisfaction. You can read more on that by Dan North and Gojko Adzic. Perhaps the biggest practical challenge that stands in the way of reaping those benefits is the burden of provisioning, installation and maintenance of requisite complex and fussy infrastructure. The recent availability of CI servers such as Jenkins & cloud-based testing services such as Sauce Labs  carries the potential to remove that barrier. This post discusses and shows how to integrate Behat, an emerging BDD framework for PHP, with Jenkins and Sauce Labs.

What is Behat?

Behat is a BDD framework for PHP. There are some tools available for BDD like Cucumber for Ruby, SpecFlow for .NET and Lettuce for Python. Behat is the first BDD tool for PHP applications. Developers can also use the  PHPSpec framework to implement classes within the Behat projects. Behat is written in PHP by Konstantin Kudryashov. With Behat, you can write human readable stories, which are used as tests to run against your application. Behat can be used for API testing, functional testing and data-driven testing. Developers will do API testing and we will carry on with functional testing (web acceptance testing) with Behat.

Functional Testing with Behat and Mink 

Behat is used for acceptance testing (any tests) by executing a Gherkin scenario. Developers can implement integrated classes. Testers start thinking of more workflow level and technical level steps (actions), which turns scenarios to features. Once a tester starts to think of Web Acceptance Testing (functional testing) with browser interaction, another tool called "Mink" comes into the picture. Mink is used for browser emulation (functional testing) where browser interactions takes place. As of now, these are the following Selenium drivers available for browser emulation.

  • Selenium provides a bridge for Selenium RC  (Selenium 1).
  • Webdriver provides a bridge for Selenium 2. (Facebook webdriver) for PHP. Currently Sauce Lab integration is not available with Webdriver.

Note: The Behat and Sauce Labs integration is currently only available for Selenium 1. Behat In Action: You must first have pear installed in order to proceed with Behat installation. Now we will run these commands from your terminal window:

$ sudo pear channel-discover pear.behat.org
$ sudo pear channel-discover pear.symfony.com
$ sudo pear install behat/gherkin-beta
$ sudo pear install behat/behat-beta

Test your installation by running this command:

$ behat --version
Behat version 2.2.0

Now let's install Mink and run the following commands from the terminal window:

$ pear channel-discover pear.symfony.com
$ pear channel-discover pear.behat.org
$ pear install behat/mink

Mink is ready to use. We have to include "mink/autoload.php" in your "bootstrap.php" file as shown below:

require_once 'mink/autoload.php';

Start Your Project Navigate to the project root directory and initialize Behat by running these commands:

$ cd /path/to/my/project

$ ls application

$ behat --init +d features - place your *.feature files here +d features/bootstrap - place bootstrap scripts and static files here +f features/bootstrap/FeatureContext.php - place your feature related code here

$ ls application features

$cd features

$ ls bootstrap

$cd bootstrap

$ls FeatureContext.php

This will create a "features" directory and "bootstrap/FeatureContext.php" for you. Now we will jump directly to the project created with a feature file. You can use NetBeans with installed Cucumber plugin for Gherkin syntax highlighting. Project structure will look like this: Directory Structure

Behat has already created a "features" directory and "features/bootstrap" directory with "FeatureContext.php" in it.

bootstrap.php

We can use this create file to define some constants and some third-party libraries that need to be included in class files. It's not mandatory to have PHPUnit and SauceOnDemand extension installed unless you wish to make PHPUnit assertions in behat tests  Make sure you have installed correct version of PHPUnit which supports SauceOnDemandTestCase extension. You can follow SauceLabs blog to install it properly. This file should look like this:

<?php
date_default_timezone_set('Europe/London');
 require_once 'mink/autoload.php';
/*
 require_once 'PHPUnit/Autoload.php';
 require_once 'PHPUnit/Framework/Assert/Functions.php';
 require_once 'PHPUnit/Extensions/SeleniumTestCase.php';
 require_once 'PHPUnit/Extensions/SeleniumTestCase/SauceOnDemandTestCase.php';
*/

behat.yml

This file is a default config file that Behat uses to execute features. An example of behat.yml is shown below:

default:
    context:
        parameters:
            javascript_session: selenium
            base_url: http://en.wikipedia.org/wiki/Main_Page
            browser: firefox
            show_cmd: open %s

You can change the drivers by changing the "javascript_session" parameter. It can be "selenium" or "webdriver to run tests locally. sauce.yml This file is used for running features on Sauce Labs. The code for this file is explained in the "Behat and Sauce Labs" section below. Note: Sauce Labs integration is available only with 'selenium' driver. build.xml This file is used for running features with ANT. We can use this ANT file to plug into Jenkins. Simple ANT file should look like this:

<project name="behat" basedir=".">
   <target name="behat">
    <exec dir="${basedir}" executable="behat" failonerror="true">
     <arg line="-f html --out ${basedir}/report/report.html"/>
    </exec>
   </target>
   <target name="runSauce">
     <delete dir="${basedir}/report" />
       <mkdir dir="${basedir}/report"/>
      <parallel>
      <antcall target="sauce"></antcall>
      </parallel>
   </target>
   <target name="sauce">
    <exec dir="${basedir}" executable="behat" failonerror="true">
     <arg line="-c sauce.yml -f html --out ${basedir}/report/report.html"/>
    </exec>
   </target>
</project>

There are two targets 'behat' and 'runSauce'. The target 'behat' can be used for running tests locally and target 'runSauce can be used to run tests on SauceLabs. report This directory is used to store HTML reports generated by Behat. The reports is stored in 'report/report.html' file. Start your Engine Remember, you have to download the latest version of Selenium server. Now navigate to the directory where you saved Selenium server .jar file. You have to launch it using command shown below:

java -jar selenium-server-standalone-2.15.0.jar

You need to start selenium server to run tests locally. In order to run tests on SauceLabs it's not required. Behat & Sauce Labs

Sauce Labs is a cloud testing service that allows you to do Selenium testing in the cloud. Sauce Labs allocates machines and browsers for your tests, captures screenshots for every step and records video of all jobs (tests). You don't need to set up separate machines to run tests. Sauce Labs helps us to write tests without complex infrastructure. In order to integrate Behat with Sauce Labs, you need to have an account with Sauce Labs. You'll need your "Username" and "API Key" to plug them into a config file. Behat executes features with "behat.yml" file by default, but we can run features with any other configuration file. We can create another configuration file like "sauce.yml" to run features on Sauce Labs. Example "sauce.yml" should look like this:

default:
  context:
    parameters:
      default_session:    goutte
      javascript_session: selenium
      base_url:           http://en.wikipedia.org/wiki/Main_Page
      browser:            firefox
      selenium:
        host: ondemand.saucelabs.com
        port: 80
        browser: >
          {
            "username":         "your username",
            "access-key":       "your API key",
            "browser":          "firefox",
            "browser-version":  "7",
            "os":               "Windows 2003",
            "name":             "Testing Selenium with Behat"
          }

We will use "sauce.yml" as a config file to run features on Sauce Labs. If you wish to run all features from the "features" directory on Sauce labs, you can use this command:

behat -c sauce.yml

When to implement step definitions?

  • If you can speak fluent Gherkin, then you don't need to write code. Behat/Mink will understand Gherkin and run your features without suggesting step definitions.
  • If the features are written by someone else, you can take full advantage of Mink's APIs in order to implement step definitions suggested by Behat/Mink.
  • It's very important to write good Gherkin to write minimum code.
  • It's very easy to access Mink API's by writing simple code.

Now we will see how you "click" particular elements on page. You need to use Xpath as a locator for that element. Mink will suggest some step definitions and you'll need to complete it like this:

/**
* @Given /^I click Something$/
*/
public function iClickSomething()
{
$this->getMink()->getSession()->getDriver()->click("//Xpath");
}

Example  : Feature wikiSearch We will write a simple feature to add a product into shopping cart. The feature will look like this:

 Feature: wikiSearch
  In order to search information on wiki
  As a Wiki user
  I want to get sensible results from site

 @javascript   Scenario Outline: Search Keywords on Google     Given I am on "/"     And I fill in searchBox with "<input>"     When I press search button     Then I should see "<output>"

    Examples:       | input       | output         |                                          | London      | lʌndən/        |       | NewYork     | nɪu ˈjɔək      |       | Sydney      | sɪdni/         |       | Mumbai      | मुंबई            |       | Bejing      | 北京           |       | Tokyo       | 東京           |       | Lahore      | لاہور            |       | Paris       | paʁi           |

Feature Explained This feature file is written in "Gherkin" DSL (Domain Specific Language). The feature file mentioned above is a good example of data-driven testing. This feature will execute our scenario for 8 different data sets mentioned in the example section. This feature will have the following steps:

  1. User enters "London" in search box.
  2. User will check if that page has city name in their local language as described in output.
  3. This test will run for 8 different cities as shown in "examples" section of feature.

This feature is also a good example of testing internationalization as it consists of test data (examples) in different languages. Now we will run this feature using command:

behat --name wikiSearch

Remember, we are running it locally for now using default config file "behat.yml" with Selenium driver. After executing the above command, we will get some step definitions suggested by Behat/Mink

Feature: wikiSearch
  In order to search information on wiki
  As a Wiki user
  I want to get sensible results from site

  @javascript   Scenario Outline: Search Keywords on Wiki # features/wikiSearch.feature:8     Given I am on "/"                         # FeatureContext::visit()     And I fill in searchBox with "<input>"     When I press search button     Then I should see "<output>"              # FeatureContext::assertPageContainsText()

    Examples:       | input   | output    |       | London  | lʌndən/   |         Undefined step "I fill in searchBox with "London""         Undefined step "I press search button"       | NewYork | nɪu ˈjɔək |         Undefined step "I fill in searchBox with "NewYork""         Undefined step "I press search button"       | Sydney  | sɪdni/    |         Undefined step "I fill in searchBox with "Sydney""         Undefined step "I press search button"       | Mumbai  | मुंबई     |         Undefined step "I fill in searchBox with "Mumbai""         Undefined step "I press search button"       | Bejing  | 北京        |         Undefined step "I fill in searchBox with "Bejing""         Undefined step "I press search button"       | Tokyo   | 東京        |         Undefined step "I fill in searchBox with "Tokyo""         Undefined step "I press search button"       | Lahore  | لاہور     |         Undefined step "I fill in searchBox with "Lahore""         Undefined step "I press search button"       | Paris   | paʁi      |         Undefined step "I fill in searchBox with "Paris""         Undefined step "I press search button"

8 scenarios (8 undefined) 32 steps (8 passed, 8 skipped, 16 undefined) 0m15.771s

You can implement step definitions for undefined steps with these snippets:

    /**      * @Given /^I fill in searchBox with "([^"]*)"$/      */     public function iFillInSearchboxWith($argument1)     {         throw new PendingException();     }

    /**      * @When /^I press search button$/      */     public function iPressSearchButton()     {         throw new PendingException();     }

As you can see from above, Behat/Mink have suggested some step definitions for undefined steps. We can implement these step definitions using Mink in "bootstrap/FeatureContext.php" file. We can implement the first step definition "iFillInSearchboxWith($argument1)" like this:

/**
     * @Given /^I fill in searchBox with "([^"]*)"$/
     */
    /*
    public function iFillInSearchboxWith($input)
    {
        $this->fillField("searchInput",$input);
    }

Now, when we run "behat" command again:

behat --name wikiSearch

You can see now sixteen step passed. The terminal window output should be like this:

Feature: wikiSearch
  In order to search information on wiki
  As a Wiki user
  I want to get sensible results from site

  @javascript   Scenario Outline: Search Keywords on wiki # features/wikiSearch.feature:7     Given I am on "/"                       # FeatureContext::visit()     And I fill in searchBox with "<input>"  # FeatureContext::iFillInSearchboxWith()     When I press search button     Then I should see "<output>"            # FeatureContext::assertPageContainsText()

    Examples:       | input   | output    |       | London  | lʌndən/   |         Undefined step "I press search button"       | NewYork | nɪu ˈjɔək |         Undefined step "I press search button"       | Sydney  | sɪdni/    |         Undefined step "I press search button"       | Mumbai  | मुंबई     |         Undefined step "I press search button"       | Bejing  | 北京        |         Undefined step "I press search button"       | Tokyo   | 東京        |         Undefined step "I press search button"       | Lahore  | لاہور     |         Undefined step "I press search button"       | Paris   | paʁi      |         Undefined step "I press search button"

8 scenarios (8 undefined) 32 steps (16 passed, 8 skipped, 8 undefined) 0m25.568s

You can implement step definitions for undefined steps with these snippets:

    /**      * @When /^I press search button$/      */     public function iPressSearchButton()     {         throw new PendingException();     }

We have to repeat this process until all steps get "passed". We can implement these steps by adding some code in "bootstrap/FeatureContext.php" file. The code will look like this:

<?php
use Behat\Behat\Context\ClosuredContextInterface,
    Behat\Behat\Context\TranslatedContextInterface,
    Behat\Behat\Context\BehatContext,
    Behat\Behat\Exception\PendingException;
use Behat\Gherkin\Node\PyStringNode,
    Behat\Gherkin\Node\TableNode;
use Behat\Mink\Behat\Context\MinkContext;
use Behat\Mink\Session;
use Behat\Mink\Driver\DriverInterface;
require_once 'bootstrap.php';

/**  * Features context.  / class FeatureContext extends MinkContext {    /*      * @Given /^I fill in searchBox with "([^"]*)"$/      */     public function iFillInSearchboxWith($input)     {         $this->fillField("searchInput",$input);     }

    /**      * @When /^I press search button$/      */

    public function iPressSearchButton()     {         $this->getMink()->getSession()->getDriver()->click("//*[@id='searchButton']");          $this->getMink()->getSession()->wait("3000");     }

After implementation, we have to run "behat" command again. You will see the feature running in the browser for all the inputs mentioned in the feature file. You will have to wait until the browser finishes all the examples. You will see the terminal output look like this:

 Feature: wikiSearch
  In order to search information on wiki
  As a Wiki user
  I want to get sensible results from site

  @javascript   Scenario Outline: Search Keywords on wiki # features/wikiSearch.feature:8     Given I am on "/"                       # FeatureContext::visit()     And I fill in searchBox with "<input>"  # FeatureContext::iFillInSearchboxWith()     When I press search button              # FeatureContext::iPressSearchButton()     Then I should see "<output>"            # FeatureContext::assertPageContainsText()

    Examples:       | input   | output    |       | London  | lʌndən/   |       | NewYork | nɪu ˈjɔək |       | Sydney  | sɪdni/    |       | Mumbai  | मुंबई     |       | Bejing  | 北京      |       | Tokyo   | 東京       |       | Lahore  | لاہور     |       | Paris   | paʁi      |

8 scenarios (8 passed) 32 steps (32 passed) 0m42.794s

We managed to get all our scenario/steps "passed" so now it's time to login into your Sauce Labs account to see this scenario running on Sauce Labs. Now we have to use "sauce.yml" config file. We will run below mentioned command from terminal and we can see output on Sauce Labs as shown below:

 behat -c sauce.yml

Screenshots for the feature running on Sauce Labs:

Building Features with Jenkins  Continuous Integration (CI) is one of the best practices in agile projects to detect bugs early. Each integration is verified by an automated build to detect integration errors as quickly as possible. Now we will see how we can integrate Jenkins to our behat project. You don't need to start selenium server to run features on SauceLabs. The HTML reports generated by Behat can be plugged directly into Jenkins. Lets see how it can be achieved. To continue, you need Jenkins installed on your machine. Now start Jenkins by executing *.war file from terminal on port 8080. You also need to visit "http://localhost:8080" to see Jenkins GUI.

$  java -jar jenkins.war
  • Visit http://localhost:8080/ and you should see Jenkins Dashboard.
  • Create New Job for behat project called "BehatSauce". Specify it as a 'Build a free-style software project'.
  • Specify your SCM. Here is Git repository
  • Select 'Invoke Ant' from 'Build' and specify 'runSauce' target
  • Specify HTML report path. (You need to have HTML report plugin installed on Jenkins)]]
  • Save this settings and Click on "Build Now"
  • Login to your SauceLabs account. You will see tests running there.
  • Sit back and enjoy your features running on Jenkins till it finishes job.
  • Few minutes later, you will see your build is "Green" and report generated as 'Sauce Report]
  • Click on 'Sauce Report' and enjoy nice looking HTML report generated by Behat.]
  • You can watch full video of job executed on SauceLabs here

Clone Source Code from GitHub

You can try these steps on your own. Source code is available on Github. Steps to follow:

  • Clone the repository from my GitHub
       $ git clone git@github.com:Shashi-ibuildings/Behat-Sauce.git
       $ cd /path to/Behat-Sauce
  • Edit 'Sauce.yml' file in order to specify your username and API key.
  • Run ANT command to execute features on Sauce Labs
       $ ant runSauce
  • Alternatively, you can configure your Jenkins job as mentioned earlier and watch your tests running on Sauce Labs.
  • You can use this configuration in your project by changing urls in 'behat.yml' and 'sauce.yml' and writing your own features.

Conclusion: We can write web acceptance tests with Behat and Mink combination. We can plug them into Sauce Labs with config file ("sauce.yml") and run them on a CI server. Now you can implement BDD practices for PHP applications by utilizing the benefits of Bahat and Sauce Labs.

Demo

To see a video demonstration of this blog post, visit on Vimeo and YouTube.  To read a little more about me, click here.

Discuss: Adding Sauce To Behat
0 Comments

Free Trial

Get access to a free 14-day trial version, or contact Sales for more information.