Selenium Tips: How to Coordinate Multiple Browsers in Sauce OnDemand

September 3rd, 2010 by Santiago Suarez Ordoñez

Let’s imagine your team is making a chat application. Nothing too fancy, just something simple in which you need to make sure multiple users get together, talk, and read each other.

Doesn’t sound that bad, huh? But how about coordinating those browsers? You need to have different sessions running at the same time in coordination and interacting with each other through your web app.

Now, when we think about parallelization and multiple browsers, Sauce OnDemand is, of course, our first answer, but here, there can be a small limitation. Sauce OnDemand has a special timeout that waits for new commands for no more than 90 seconds*. If it doesn’t get any new command, it will kill the job and assume something went wrong.

This effects us, as we need to get multiple browsers running in parallel and in coordination. As we request them one after another, depending on the amount of browsers and the load our service is put under, the first ones might timeout while waiting for the others to come.

To avoid this, the best solution is to write a multi-threaded script, in which browsers will send heartbeat commands while they are waiting for the rest. Multi-threading can be done with every programming language, but I decided to write my example in Python because I think it’s understandable for almost anyone, even if you’ve never seen Python before. The complete example is in github as a gist, and here is a brief description of what each interesting part is doing:

def get_browser_and_wait(browser, browser_num):
    print "starting browser %s" % browser_num
    browser.start()
    browser.open("/") # if we get here, OnDemand already gave us the browser
    browsers_waiting.append(browser)
    print "browser %s ready" % browser_num
    while len(browsers_waiting) < len(browsers):
        print "browser %s sending heartbeat while waiting" % browser_num
        browser.open("/")
        time.sleep(3)

get_browser_and_wait is the function that will take care of the ‘start the browser’ request in OnDemand and wait till it comes back. Once the server is ready, it will keep sending open commands as heartbeats and waiting until the other requested browsers are ready and waiting too.

thread_list = []
for i, browser in enumerate(browsers):
    t = Thread(target=get_browser_and_wait, args=[browser, i + 1])
    thread_list.append(t)
    t.start()

for t in thread_list:
    t.join()

This is the magic multithreading part, in which we iterate over every Selenium instance and call get_browser_and_wait with it. Once we send each browser to a thread, the main thread will continue and get to the t.join() part. By this method, the main thread will wait for every browser thread to complete in order to proceed with the rest of the code.

Again, the full example is up in github: http://gist.github.com/511658. If you’re interested in learning what you can use this kind of test for, stay tuned!

* This helps avoid running (and charging) tests that get disconnected or crash.

  • Share/Bookmark

Introducing selbot

August 17th, 2010 by Miki Tebeka

Sauce Labs is based on open source and we use a lot of it. We also do our best to contribute back to the community by working on Selenium, organizing #SFSE meetups, answering questions on the messages boards, and more.

Today we are happy to announce selenium.saucelabs.com, a searchable archive for the Selenium IRC channel. selenium.saucelabs.com is powered by “selbot,” and it hangs out at #selenium on freenode.

“selbot” has two main functions:

“selbot” is made of open source components, mainly Supybot and xapian. An open source project itself, you can find the code on github.

“selbot” is a humble start, and we have great plans for it in the future. For now, log in to #selenium on freenode and have a look. And while you’re there, answer a few questions – it’s a great way to give back to the community.

  • Share/Bookmark

Setting up Cucumber + Webrat + Selenium

June 15th, 2010 by Sean Grove

There’s quite a bit of information out there on getting these disparate tools to work together, but a great deal of it is out of date. To clear things up a bit, I’ve documented all the gems and modifications necessary to get these pieces of open source software up and running together. As time goes on, I’ll be expanding this blog post with notes about the pitfalls and various platform issues that may be discovered (I’m looking at you, Snow Leopard), but this should get most people up and running right away.

This is all using a clean REE environment via the poorly named but wonderfully written rvm, or Ruby Version Manager.

Gems you’ll need:

gem install actionmailer actionpack activerecord activeresource activesupport builder cgi_multipart_eof_fix cucumber cucumber-rails daemons database_cleaner diff-lcs fastthread gem_plugin gherkin json json_pure mime-types mongrel net-ssh net-ssh-gateway nokogiri rack rack-test rails rake rdoc rest-client rspec rspec-rails Selenium selenium-client sqlite3-ruby term-ansicolor trollop webrat

Some of those are not strictly necessary, but simply nice to have, while others solved some unexpected problems with the bare necessities. I’ll prune this list as feedback comes in from people’s experiences.

Points to watch out for:
Nokogiri: This was easily the worst on my Snow Leopard machine. It relies on the native libxml2, which had problems with 32/64 bit compatibility. No matter what I tried, errors kept coming up. I had to clean everything out with my MacPorts installation and force a universal installation.
Webrat: The Selenium server jar that was included by default caused no end of headaches. I had to manually go in to the directory, remove the default server jar, and download the newest version from the seleniumhq download page.

Once those are set up, you should be able to use Cucumber, Webrat, and Selenium together without too much headache. Then you can refer to our webinar video (which will be posted to the blog later this week) to learn how to use Cucumber to easily run Webrat and Selenium.

  • Share/Bookmark

Selenium IDE 1.0.7 is released!

May 27th, 2010 by Adam Goucher

Two months after 1.0.6 was released, Selenium IDE 1.0.7 is now available for general release.

If you are using version 1.0.4 or older, you will want to download Selenium IDE 1.0.7 now. For newer version users, the 1.0.7 update will be pushed to you the next time Firefox checks for plugin updates. (You can force this by clicking Tools, Add-ons, Find Updates in Firefox.)

By far the most exciting new feature is the ability to drag-and-drop within the Selenese editor. No longer do you have to cut your command, insert a new row, reinsert it because you forgot whether it goes above or below, then paste. Now you just move the click on the command and move it to where you want it. Same thing for ordering your tests in the suite. A big thanks to Jérémy Hérault for that feature.

Other than that, there was a slight tweak to the plugin API and localization into Swedish.

So a small release in terms of features, but an exciting one.

To see the full release notes and who contributed what, visit the Google Code wiki. Also on the Google Code site is the Issue Tracker for any problems you find. (Just make sure you tag them as ide.)

  • Share/Bookmark

Selenium Tips: Working with multiple windows

May 24th, 2010 by Santiago Suarez Ordoñez

One of the most common issues Selenium users look to Sauce Labs for assistance on is how to deal with popup windows, or even how to create your own new windows to do something in a shared session that could be connected with the main flow of your test.

In this week’s tip, we’re going to explain how to handle new windows to run tests like this one recorded with our cloud hosted Selenium service, Sauce OnDemand.


For multiple window tasks, Selenium provides the following commands:

  • openWindow(url, windowId): Will open your new window
  • waitForPopup(windowId, timeout): Will wait for the window to be opened and loaded
  • selectwindow(windowId): Will focus on this windows and any future actions will act on it

So for example, we could create the above test using Sauce IDE to run on http://google.com:

open /
type q sauce labs
click btnG
waitForTextPresent Sauce Labs
openWindow /search?q=santiago santiago
waitForPopUp santiago 30000
selectWindow santiago
assertTitle santiago – Google Search
selectWindow null
assertTitle sauce labs – Google Search

However, due to an active bug, IDE won’t be able to play back that test as written. To fix that, we’ll export this script to a programming language like Java and run in on Sauce OnDemand.

selenium.open("/");
selenium.type("q", "sauce labs");
selenium.click("btnG");
for (int second = 0;; second++) {
if (second >= 60) fail("timeout");
try { if (selenium.isTextPresent("Sauce Labs")) break; } catch (Exception e) {}
Thread.sleep(1000);
}
selenium.openWindow("/search?q=santiago", "santiago");
selenium.waitForPopUp("santiago", "30000");
selenium.selectWindow("santiago");
assertEquals("santiago - Google Search", selenium.getTitle());
selenium.selectWindow("null");
assertEquals("sauce labs - Google Search", selenium.getTitle());

The script will work as expected, opening the window and doing the parallel work that you expected it to do.

Notice that at the end of the script, we are using selectWindow(“null”), which means that we want to focus on the main window back.

Did you find this tip useful? Need some help adapting this to your needs? Let us know in the comments!

Until next time, happy testing!

  • Share/Bookmark

Selenium participating in Google Summer of Code (GSoC)

April 6th, 2010 by John Dunham

Selenium has been accepted into the Google Summer of Code program.  This means participating students will get $5000 USD to hack on Selenium-related Open Source projects in the summer, while getting mentored by contributors of the Selenium project.   The deadline for application is April 9, 2010 and Selenium still has open slots -  from writing python bindings for Selenium 2.0 to writing mobile support for Android and iPhone to Selenium IDE 2.0.  You can suggest projects as well.  Once you have submitted your application, you will have until April 18, 2010 to fine tune your proposal with our mentors.  Google will announce the list of accepted students / proposal by April 26, 2010.

You can find more information here:
http://code.google.com/p/selenium/wiki/GoogleSummerOfCode

Link to apply:
http://socghop.appspot.com/gsoc/student/apply/google/gsoc2010

If you know students (or people that have children :-) attending college, please pass this info along.

This is a great opportunity for students to work on open source projects that’s used by millions of developers and QA professionals, gain skills in Selenium and ultimately help drive higher quality software everywhere.  Do note that most of the work will be done off-site, however, if you are paired up with mentors in San Francisco, LA, Portland, London…  you may have a chance to work with them directly too.

  • Share/Bookmark

Running your Selenium tests in parallel: Perl

April 1st, 2010 by Miki Tebeka

The Test::WWW::Selenium does a good integration of Selenium with Perl testing framework. However by default, tests runs in sequence. One easy solution is to use the Thread::Pool::Simple module to run the test functions in parallel. Short testing showed that this parallel version ran for 19 seconds vs 39 seconds for the serial run.

#!/usr/bin/env perl

use Thread::Pool::Simple;
use Test::More "no_plan";
use Test::WWW::Selenium;

sub test_page() {
    my ($url, $input, $button) = @_;

    my $sel = Test::WWW::Selenium->new(
        host => "localhost",
        port => 4444,
        browser => "*firefox",
        browser_url => $url,
    );

    $sel->open($url);
    $sel->type_ok($input, "Sauce Labs");
    $sel->click_ok($button);
    $sel->wait_for_page_to_load_ok(5000);
    $sel->body_text_like(qr/Selenium/);
    $sel->close();
}

sub test_google() {
    &test_page("http://google.com", "q", "btnG");
}

sub test_yahoo() {
    &test_page("http://yahoo.com", "p", "search-submit");
}

sub test_bing() {
    &test_page("http://www.bing.com", "q", "go");
}

sub worker() {
    my $func = shift;
    # We can't pass CODE items to a pool, so we use eval
    eval("&$func();");
}

my @names = (
    "test_google",
    "test_yahoo",
    "test_bing",
);

my $pool = Thread::Pool::Simple->new(
    min => 3,
    max => 10,
    do => [\&worker],
);

foreach $name (@names) {
    print "Adding $name\n";
    $pool->add($name);
}
$pool->join();

Converting the code to work with Sauce OnDemand is a breeze as well, we will also use the job-name attribute to assign name for our jobs.

#!/usr/bin/env perl

use Thread::Pool::Simple;
use Test::More "no_plan";
use Test::WWW::Selenium;

# Replace username and access-key with your real
# Sauce OnDemand username and access-key
my $settings = qq(
{
    "username" : "YOUR-SAUCE-USER-NAME",
    "access-key" : "YOUR-SAUCE-ACCESS-KEY",
    "os" : "Linux",
    "browser" : "firefox",
    "browser-version" : ""
    "job-name" : "_JOB_NAME_"
}
);

sub test_page() {
    my ($name, $url, $input, $button) = @_;

    my $opts = $settings;
    $opts =~ s/_JOB_NAME_/$name/; # Add job name

    my $sel = Test::WWW::Selenium->new(
        host => "ondemand.saucelabs.com",
        port => 80,
        browser => $opts,
        browser_url => $url,
    );

    $sel->open($url);
    $sel->type_ok($input, "Sauce Labs");
    $sel->click_ok($button);
    $sel->wait_for_page_to_load_ok(5000);
    $sel->body_text_like(qr/Selenium/);
    $sel->close();
}

sub test_google() {
    &test_page("Google", "http://google.com", "q", "btnG");
}

sub test_yahoo() {
    &test_page("Yahoo", "http://yahoo.com", "p", "search-submit");
}

sub test_bing() {
    &test_page("Bing", "http://www.bing.com", "q", "go");
}

sub worker() {
    my $func = shift;
    # We can't pass CODE items to a pool, so we use eval
    eval("&$func();");
}

my @names = (
    "test_google",
    "test_yahoo",
    "test_bing",
);

my $pool = Thread::Pool::Simple->new(
    min => 3,
    max => 10,
    do => [\&worker],
);

foreach $name (@names) {
    print "Adding $name\n";
    $pool->add($name);
}
$pool->join();
  • Share/Bookmark

Selenium IDE 1.0.6 is released!

March 26th, 2010 by Adam Goucher

Selenium IDE 1.0.6 is now available. For those of you running 1.0.5, the update will automatically be pushed to you over the next couple days, and the rest will have it once we point AMO at the installer. If you are new to Selenium IDE then you can…

Download Selenium-IDE 1.0.6 now

The most important bug fix this release is the log message that was displayed in red (scary!) when executing an open command. Other than that, it was mainly just some incremental stuff. Here are the top 3 changes in terms of what I think most people care about.

  • Deal with new Open semantics – Under normal usage, the open command takes just a Target, but now you can also give a Value. If you provide anything other than ‘true’ it will do an AJAXy HEAD request to the server to determine whether the resource is on the server. There was a bit of an issue with the on/off logic that was causing this to be triggered inappropriately resulting in the error message and in some cases errors on the web server (when they had not configured how to deal with HEAD requests). I believe we have fixed this.
  • Type ahead – Some people using Firefox 3.6.x were seeing errors when using the command type ahead. That should work without error now.
  • Default Record Status – It always bugged me that Se-IDE was recording as soon as it opened. There is now a preference for controlling that behaviour

To see the full release notes and who contributed what, visit the Google Code wiki. Also on the Google Code site is the Issue Tracker for any problems you find. (Just make sure you tag them as ide.)

  • Share/Bookmark

Selenium Tips: Finding elements by their inner text using :contains, a CSS pseudo-class

March 19th, 2010 by Santiago Suarez Ordoñez

As we already mentioned in our previous posts CSS Selectors in Selenium Demystified and Start improving your locators, CSS as a location strategy generally far outperforms XPATH.

That’s why for this Tip of the Week, we are presenting our users one more CSS pseudo-class to keep moving their test suites to this faster and cleaner location strategy.

Let’s imagine you have an element like the following:

<div>Click here</div>

The easiest way to locate it is using it’s “Click here” inner text and in XPATH you would do it using XPATH’s text() function:

//div[text() = "Click here"]

or even better (because the previous version can sometimes fail depending on the browser):

//div[contains(text(), "Click here")]

Now, if you want to move your locators to CSS in situations like this one, all you have to do is use the :contains() pseudo-class, with which your tests would end up using the following locator:

css=div:contains("Click here")

Did you find this useful? Are you still fighting with another XPATH functionality to completely move your tests to CSS? Let us know about it in the comments.

  • Share/Bookmark

Selenium Tips: Parametrizing Selenese tests

March 12th, 2010 by Santiago Suarez Ordoñez

Even though I believe Selenium IDE should just be a transition environment for new Selenium users to reach Selenium RC, we know a lot of our users keep all their tests in Selenese and run them using Selenium IDE as their main testing tool.

That’s why at Sauce we have developed Sauce IDE, our own extension of Selenium IDE that combines Selenium RC’s cross-browser capabilities with the simplicity and interactive design of Selenium IDE.

To continue empowering the IDE users, I’m writing this tip of the week that will help making Selenese tests easier to maintain and write.

As you all know, Selenese is not a real programming language. Variable storage in it is pretty rough and there isn’t an easy way to share information between different Test Cases in the same Test Suite.

However there is a way to do it, through the use of the user-extensions.js file. Let me show you how in two easy steps:

First, create a user-extensions.js file with the following content:

storedVars["url"] = "/staging/search";
storedVars["title"] = "Search Your Item";
storedVars["search-string"] = "234234kjkj";

Then open Selenium IDE, or even better Sauce IDE, and in the Options menu, select the file in the “Selenium Core Extensions” field.
Close and open Selenium IDE again and you should be ready to use that variable in any test that you write. Even better, throughout your whole test suite.

For example:

open ${url}
assertText h1 ${title}
type search-field ${search-string}
click btnSearch
assertTextPresent Search Results for ${search-string}

This may not look super useful at first, but once your test suite grows, keeping things that change from time to time in a single place, makes it really easy to maintain, so if your application’s URL changes from “/staging/search” to “/search”, you will only need to fix the user-extensions.js file and all the hundreds of tests you could have will pass the same way as they did before.

Check the wikipedia’s page on DRY for more information about this programming technique.

Hope you found this useful. If you did, please let us know in the comments.

Santi

  • Share/Bookmark