Getting Started With Web Testing Using Selenium & Sauce Labs

February 9th, 2012 by Will Iverson

This is a guest blog post by Will Iverson, David Drake, Osama Khalaf, and Adam Herbst of Dynacron Group, a technology consulting firm based outside Seattle that provides software development (Java/.NET) and BI/DW services based on Continuous Delivery.

Dynacron Group staff started working with Selenium nearly four years ago. The biggest problem we had was execution time – we had a limited number of browsers and during launches, test & release jobs would start to pile up as we were waiting for the limited browsers in our homebrew Selenium grid to be available.

To solve that problem for a new project we started almost a year and a half ago, we decided to use Sauce Labs. To make it as easy as possible for the test team, we created a parallel execution framework to make it dead easy to run the tests, and a template project to make it dead simple to get started.

By standardizing the parallel execution framework and template project, a tester can pretty much just sign up with Sauce Labs, follow a few basic configuration steps, and start writing test suites that leverage parallel execution in no time. Most people start with the Selenium Recorder, just pasting commands in from the clipboard to start and then pretty quickly graduate to just writing stuff in their IDE of choice.

Let me explain why parallel execution is so cool: we write 200 tests. We run those tests in parallel across five browsers (Firefox, Chrome, Safari, and two versions of Internet Explorer). That’s a thousand tests. If we ran those serially, one after the other, it would take the better part of a day. By running fifty parallel browsers, we can finish that test suite in fifteen minutes.

The Sauce Labs site can give you an overview of the other awesome features (recording video of every job, step-by-step screenshots, and Sauce Connect to name a few) – but nothing is as nice for us as the time-to-market advantage.

For more information, check out:

Tutorial

Just download this tutorial and follow the instructions to get started.

webtest-quickstart

Webtest Quickstart is the template project. This is where you’ll want to go to get started quickly.

If you’re just starting on a new Selenium test project, you might consider using JUnit to manage your test cases. The webtest-quickstart project can provide a simple starting point for importing your IDE tests into JUnit and running them in maven.

To get started, just download the the webtest-quickstart project from github, or use the maven archetype from the same site. Inside the project, you’ll find sample test cases for many different testing situations. There are samples for running tests against a site running on your computer, or against websites on your network. Best of all, the parallel-webtest library gives you the ability to easily configure how tests are run. The project pom has sample profiles for running your tests in your own browser or in several different Sauce browsers in parallel, all from the same test code.

parallel-webtest

Parallel Webtest is the library that handles the parallel execution & configuration.

At its heart is a JUnit base test class that manages the browser lifecycle. It configures and launches the browser before executing a test class, and cleans it up afterwards. This isolates the test code, promoting cleaner test methods. Browser selection and parallel configuration are handled with a few configurable system properties; the same test code can be run in serial or parallel, locally or remotely, against one browser type or many. As a result, a test suite that once took four hours to run can be finished in as little as fifteen minutes using the same build server.

The webtest-quickstart and webtest-quickstart-archetype are open-source. Both are available on Dynacron Group’s Github account. If you have a chance to try it out, let us know what you think at info@dynacrongroup.com. And happy (faster) testing!

Share

GivenWhenThen: Simple, Powerful Acceptance Testing for Node.js

January 25th, 2012 by Doug Wright

Over the last year or so, interest in and development for Node.js has accelerated dramatically. There is no shortage of excitement and promise in this burgeoning community and for many of the same reasons cited by others, about six months ago we decided it made sense to dive in ourselves.

Coming from a committed testing background on multiple web frameworks, including Rails, we are strong adherents to BDD. Naturally the first thing we did when spinning up on the Node stack was look around for good BDD acceptance testing tools. Not finding anything that fit for us, we went ahead and rolled our own. In this post, we’ll introduce GivenWhenThen, a DSL and runner that allows anyone to easily construct acceptance tests with BDD semantics in straightforward, sentence-like statements and then run them against Sauce Labs.

Writing a Story

Although you can also directly access the Selenium RemoteWebDriver that is under the covers, GivenWhenThen is primarily intended to provide a DSL for writing executable stories in the Dan North format. To write a story, begin with a BDD description in a xxx_test.coffee file:

 

story 'Executing a Google search',
  """
  As a human
  I want to perform a search
  So that I can access the world's information
  """, ->

 

And then add one or more scenarios:



scenario "Search for info about Node.js", (browser) ->
  browser
    .given "I am on the homepage", ->
      browser.step(steps.visitHomepage)
    .when "I enter search terms", ->
      browser.typeInElement('q', 'nodejs', using:'name')
    .and "submit the search", ->
      browser.clickElement('btnG', using:'name')
    .then "I see search results", ->
      browser.assertTextPresent('results')
    .and "the results contain information about nodejs", ->
      browser
        .assertTextPresent('node.js')
        .assertTextPresent('nodejs.org')


 


Each scenario has “given“, “when“, and “then” steps.




    

  • Given: Set up the initial conditions for the scenario.
    • 

  • When: Take the action the scenario is testing.
    • 

  • Then: Assert the conditions expected after taking the tested action.
    • 



Each step contains one or more chained calls to WebDriver commands in the form of browser.someCommand.


Each step (given, when, then) can have an arbitrary number of and steps following it (see above example).


Steps


Often there are steps that are repeated throughout many scenarios, for example “visit homepage” or “sign in”. This kind of functionality can be defined in steps and referred to in scenarios via browser.step():




 


scenario "Search for info about Node.js", (browser) ->
  browser
    .given "I am on the homepage", ->
      browser.step(steps.visitHomepage)


 


Steps are defined in *_steps.coffee files. Multiple steps per file can be defined as follows:


 


steps.visitHomepage = (browser) -> browser.get 'http://www.google.com'


 


Multiple steps files can be created to organize your steps in a manageable way.


Configuration


Configuration is simple and contains:




    

  • Overall story and Sauce Labs configuration.
    • 

  • Browser / OS definitions. Stories will be run against each browser/os configuration defined.
    • 



For example:


config.credentials =
  'username':   'sauce_labs_username'
  'access-key': 'sauce_labs_access_key'

config.settings =
  'max-duration': '180'

config.browsers = [
  {
    'platform':     'VISTA'
    'browserName':  'firefox'
    'version':      '7'
  }
  {
    'platform':     'LINUX'
    'browserName':  'firefox'
    'version':      '7'
  }
]


 




Subtitles In Test Videos


One of our favorite features of Sauce Labs is test videos. It is incredibly useful to be able to view a video of a failed test and quickly understand what went wrong. However, one issue that has been raised about the videos is that they could sometimes benefit from some context about what is happening on screen.Well, it just so happens that this context is conveniently codified in the steps of BDD scenarios! So, by flipping a switch in GivenWhenThen, you can add BDD subtitles to your Sauce test videos. When the subtitle flag is passed to the GivenWhenThen runner, subtitle divs will be inserted into the test browser, which will show up in the Sauce Labs test videos and illustrate which of the BDD steps is currently being executed:




 







 


Design Background




In addition to being built on top of Sauce Labs and Selenium 2, GivenWhenThen is heavily influenced by the BDD movement.


While it is also clearly influenced by Cucumber, aficionados will surely notice differences. Most obviously, in GivenWhenThen there is no prose-to-code translation step. Cucumber uses the Gherkin language, which parses the prose into features, scenarios, and steps.


In GivenWhenThen, the DSL is embedded directly in the host language. We do realize there is real benefit to the full Cucumber vision – including Gherkin – but we took a leaner approach to getting the BDD structure without completely porting Cucumber.  To us, the core power of the BDD approach comes from the conceptual framework Dan North developed. Just stating the story and scenario in that formal way and using the “given, when, then” flow gives powerful structure and coherence to your specifications.






Conclusion


GivenWhenThen fills a need we have by providing a Selenium 2 driver and Sauce Labs integration for Node.js, and adding BDD structure to our tests, while staying light and keeping us close to the host language. We hope you find it useful as well!


Share
				


                

                    

                       Posted in BDD, Guest Posts                    

                    
                    

                


			


				

				
Selenium Client Factory for Python

            	January 23rd, 2012  by Ross Rowe 

				

					
Integration for Selenium testing running with your continuous integration builds with Sauce OnDemand and Sauce Connect is made easy through the plugins we’ve developed for Bamboo and Jenkins.


These plugins allow you to specify the browser to be used by your tests and launch Sauce Connect prior to the running of your tests.


They set several environment variables that include the settings for your build, and your tests will then need to reference these environment variables as part of the Selenium setup.


However, explicitly referencing these variables can make your test code cluttered and it can be cumbersome to get the tests structured so that they work against either a local browser or Sauce OnDemand.


Rather than updating your code to explicitly reference the environment variables the plugins set, you can instead use the Selenium Client Factory library.


This library will read the environment variables set by the CI plugin, and instantiate a Selenium/WebDriver instance that will run your tests against Sauce OnDemand.


Selenium Client Factory is a Java library, which can be incorporated into your tests by referencing the jar file or if you’re using Maven, by including the following dependency:


<dependency>
    <groupId>com.saucelabs.selenium</groupId>
    <artifactId>selenium-client-factory</artifactId>
    <version>2.1</version>
    <scope>test</scope>
</dependency>


While this is great for Java applications, Selenium and the CI tools themselves support a number of languages.


One of our users has developed a Python implementation of the Selenium Client Factory code.


The factory object reads environments variables setup by the Bamboo plugin and creates a remote Sauce OnDemand session accordingly. If the Sauce-specific environment variables aren’t set, the library will create a local Selenium configuration.


In addition, the factory will output the Sauce OnDemand Session Id to the build output, and the Sauce CI plugins will parse this id and associate the Sauce Job with the CI build.


- The Selenium Client Factory library makes integrating your tests with Sauce OnDemand and your continuous integration build tool easy – you can configure your local tests to run against a local browser, and use your CI environment to access Sauce OnDemand, all via properties and without having to change any code.


To use Selenium Client Factory under Python, just instantiate Selenium via:


from SeleniumFactory import *

#For selenium 2 webDriver:
webDriver = SeleniumFactory().createWebDriver()

#For selenium 1 RC:
browser = SeleniumFactory().create()


The code for the Java and Python implementations is open source, so if you’d like to implement the Selenium Client Factory in your language of choice, either take a look at the code and have a go, or let us know and we can take a look at it.


Share