Selenium IDE 1.0.5 is released!

February 19th, 2010 by Adam Goucher

One month (to the day!) after the release of Selenium IDE 1.0.4 was released, 1.0.5 is now available.

Download Selenium-IDE 1.0.5 now

This release fixes a couple really annoying bugs that were introduced in 1.0.4 and starts to build towards the future. Here are the highlights.

  • Self-hosting – Se-IDE will not longer be relying on addons.mozilla.org to be hosting updates. This gives the project more flexibility around when releases are available to users.
  • Back into main project – The code for Se-IDE was in a separate subversion branch than the rest of the current Selenium project which meant its visibility to other developers was lower and that changes were not easily propagated. Se-IDE now lives in the main project so will reap the benefits of the rapidly changing core.
  • User Formats work again – In 1.0.4 there was a bug which meant that while you could add a custom format it would not appear in the list. This has been fixed.
  • Version Display – The version of Se-IDE now appears in the title bar
  • Reload Extensions – A tweak was made to the ‘reload user extensions’ functionality introduced in 1.0.4 so the toolbar button only appears when you have turned it on.
  • Bool Preferences – Preferences used to only be string values, even when a boolean would have been a better choice. Well, now you have a choice about whether you want your preference to be a string or bool.
  • Plugins Pane – There is a new pane in the Options screen called ‘Plugins’. This will become the central means of managing plugins in future releases, but for now it doesn’t do much aside from listing which plugins have registered themselves with the (also new) addPlugin(id) API function.
  • Icons – You will see a couple icons now in Firefox for Se-IDE rather than the default ones. Such as in the main Extensions window and in the Tools menu
  • For the full release notes and who contributed what, see the Google Code wiki. Also on the Google Code site is the Issue Tracker for any problems you find. (Just make sure you tag them as ide.)

    And as before, thanks to Sauce Labs for sponsoring my work on this release.

    -Adam Goucher

    Share

Sauce RC 1.0 Released with Web UI for Selenium RC!

February 12th, 2010 by Santiago Suarez Ordoñez

We’re pleased to announce we just released the new Sauce RC!

Hoping you all find this useful, we created a web interface for anyone to configure Sauce RC from the browser, on the same machine or even remotely!

Here’s a video of Sauce RC in action. We hope you download and enjoy it!

Share

Running Selenium RSpec tests for Rails 2.3.5

February 9th, 2010 by The Sauce Labs Team

This article will get you setup with the bare-minimum environment to run Selenium tests with RSpec, for automated, full-stack testing of Rails apps. Once that’s done, we’ll work on polishing it for a nicer experience. In this post you find amalgamations of mostly outdated articles from around the internet updated to work with a modern rails system.

It’s written on a clean environment courtesy of rvm, so you may have some of the gems already setup.

A new Rails app, RSpec, and rspec-rails

First off, we’ll start with a new rails project:

gem install rails --no-ri --no-rdoc
rails rspec_saucerc
gem install rspec
gem install rspec-rails

Edit config/environment.rb and add:

  config.gem "rspec", :lib => false, :version => ">= 1.2.9"
  config.gem "rspec-rails", :lib => false, :version => ">= 1.2.9"

This sets up all the required files for RSpec to get off the ground, but we need to integrate it into Rails:

script/generate rspec

That will add the rake tasks, create the appropriate directories, and basic files. Now let’s get Selenium up and running!

Selenium

We’ll need to create a separate selenium database. in config/database.yml, let’s add:

selenium:
  adapter: sqlite3
  database: db/selenium.sqlite3
  encoding: utf8
  timeout: 5000

To get Selenium running under our RSpec stories, we’ll need the Selenium gem (note the capital “S” — it’s case-sensitive). Let’s also make sure we have the sqlite3 gem installed, and then prepare an appropriate environment for our selenium tests:

gem install Selenium
gem install sqlite3-ruby
cp config/environments/test.rb config/environments/selenium.rb

Open config/environments/selenium.rb and remove the last line that reads:

config.gem 'rspec-rails', :version => '>= 1.3.2', :lib => false unless File.directory?(File.join(Rails.root, 'vendor/plugins/rspec-rails'))

Sauce RC and Selenium RC

Sauce RC will broker all communication between rails and any browsers you might be driving. Get it from our downloads page for Windows or Mac, or use Selenium RC for Linux, and start it up so we can run our tests.

Testables

We’ll need some fodder to test. Use the RSpec generator, then create and migrate the database:

script/generate rspec_scaffold person name:string age:string language:string
rake RAILS_ENV=selenium db:create
rake RAILS_ENV=selenium db:migrate

Startup our test server

Rails tests do not normally bind to a webserver, so there is no way for Selenium to access the frontend. As a temporary workaround, we’ll manually invoke a Rails server using the selenium environment:

script/server -e selenium

Our spec helper

We will be using the Selenium gem, so let’s add it to the spec helper:

gem "selenium-client"
require "selenium/client"
require "selenium/rspec/spec_helper"

Example story

We’ll gloss over which types of tests should include selenium front-end testing for now, and just say it belongs in integration tests. Let’s run an example story that will fail the first time through, and we’ll then fix it.

Put the following in spec/integration/people_spec.rb:

require 'spec_helper'
describe "People" do
  before(:all) do
    @verification_errors = []

    @browser = Selenium::Client::Driver.new(
      :host => "localhost",
      :port => 4444,
      :browser => "*firefox"
      :url => "http://localhost:3000",
      :timeout_in_second => 90)

    @browser.start
  end

  before(:each) do
    @browser.start_new_browser_session
  end

  append_after(:each) do
    @browser.close_current_browser_session
    @verification_errors.should == []
  end

  it "should create a new Person with valid input" do
    @browser.open "/people"
    @browser.click "link=New person"
    @browser.wait_for_page_to_load "2000"
    @browser.type "person_name", "Jason Huggins"
    @browser.type "person_age", "26"
    @browser.type "person_language", "Albanian"
    @browser.click "person_submit"
    @browser.wait_for_page_to_load "30000"
    @browser.is_alert_present.should be_true
  end
end

Let’s try it out:

rake spec:integration
F
1) 'People should create a new Person with valid input' FAILED
expected false to be true
./spec/integration/people_spec.rb:22:

Finished in 7.443904 seconds

1 example, 1 failure

It’s failing as we expected it to (there shouldn’t be a javascript alert on submit). Let’s change the last line:

    @browser.is_alert_present.should be_false

… and try our test again:

rake spec:integration
.

Finished in 7.079062 seconds

1 example, 0 failures

Looks good! We finally have a working rails project with rspec runners and Selenium.

Weaknesses

Although we’ve got Rails, RSpec, and Selenium all working together, the relationship is not harmonious.

Automated tests in rails are not meant to serve the outside world, and as such they don’t bind to a webserver/port. Tests are run within the same process, which makes them quite fast, but prevents us from using browser-based frontend tools such as Selenium. To get around that, we manually fired up a rails server instance with the selenium environment, but this is messy for a number of reasons:

  • We have to manually start/stop the test server that selenium wants to access
  • We have to manually create the selenium environment’s database
  • We have to manually migrate the selenium environment each time there’s a schema change
  • Worse, we have to manually reset the database after each run
  • Running Selenium tests in serial is slow

We‘ve developed a plugin to automate some of these issues, called SpecStorm. We’ll go over installing it to get the most out of your tests (including running them in parallel) in the next post.

Notes:

Selenium matchers are case-sensitive: @browser.click “link=New Person” won’t match a link with “New person”

Share

Sauce OnDemand supports Firefox 3.6

February 3rd, 2010 by The Sauce Labs Team

Following our zero-day support for Firefox 3.6 in Sauce RC, we are pleased to announce that Sauce OnDemand now supports Firefox 3.6 as well.

Use the following settings in your JSON configuration:

{
    "username" : "SAUCE-USER-NAME",
    "access-key" : "SAUCE-API-KEY",
    "os" : "Windows 2003",
    "browser" : "firefox",
    "browser-version" : "3.6."
}
Share

Death to Internet Explorer! Long Live Internet Explorer!

February 2nd, 2010 by Jason Huggins

There has been a lot of negative press surrounding Internet Explorer. This week, Google sent an email to Google Apps administrators explaining their intentions to phase out support for Microsoft Internet Explorer 6.0. Just last month, French and German governments asked citizens to stop using the Internet Explorer browser (no matter the version). Yes, there is even a blog titled “Why IE Sucks.” (Parental discretion advised).

We’re of mixed opinion on the matter. Do we believe people should dump Internet Explorer? Yes! In fact, I outline three reasons why below. At the same time, due to subpar browsers like Internet Explorer, the value of cross-browser functional testing using Selenium is that much more important and relevant. Call it job security for testers.

I have no secret reason to pan Internet Explorer. I work closely with all browser flavors including Internet Explorer, Firefox, Safari, Chrome and Opera. Though I am a former Googler, I have no built-in bias to push one browser over another. And even though I recommend users not use IE, Sauce Labs supports and will continue to support IE in our downloadable products, Sauce IDE and Sauce RC, and our cloud testing service, Sauce OnDemand. With that context, here are three reasons why I feel you should dump Internet Explorer.

1) IE is a virus magnet
Virus authors target IE/Windows users because IE still has the majority market-share. If they targeted Opera users, even if they had 100% success rate, they’d still not have a massive impact. In this case, following the herd can get you more hurt. Meanwhile, creators of Firefox and Chrome argue that their browsers are more secure than IE in that their open source nature makes it easier for security analysts to inspect the source code and fix security holes. Of course, the best way to test that argument will be when Firefox or Chrome reaches majority market share  (already true in some places) and virus authors target those browsers, too.

2) IE is slower
IE 6,7,8 are all significantly slower compared to Firefox, Chrome, and Safari when viewing very dynamic sites (like Google Maps or Gmail). Google Chrome is arguably the fastest. Among the “alpha geek” set, Google Chrome (and the non-Google branded Chromium version) is becoming their every-day browser because it is so fast.

3) Web-standards and innovation
MS is becoming irrelevant to the conversation. Microsoft is not keeping up, so users are missing out on new features. HTML5 brings offline storage, and new kinds of applications that leverage the new <canvas> and <video> tags. Microsoft is moving so slowly that web developers are more often ignoring IE to focus on Firefox, Chrome, and Safari when building bleeding-edge applications. IE users will be left behind for these new applications. This is especially true on mobile devices, where the WebKit rendering engine is the de facto standard. On the desktop, Chrome and Safari are built on WebKit and are benefiting in new features and bug fixes because of the innovation going on in the mobile space.

Share