Posts Tagged ‘ruby’

Video Tutorial: Using Cucumber to Test Apps in the Cloud

June 25th, 2010 by Ashley Wilson

Sauce Labs recently hosted the first in a series of webinars devoted to new Sauce OnDemand product features. Sean Grove, a developer at Sauce Labs and our resident Ruby expert, showed us how to set-up a Cucumber application, which many attendees were interested in learning more about. Cucumber is an easy-to-use testing tool that lets you write code in a language nearly identical to plain English, allowing even non-programmers to understand what is being tested.

The second half of Sean’s demo had him configuring Cucumber to switch between Webrat (the default testing tool) and Selenium. Then he ended with a overview of why Sauce OnDemand is the best platform for using Cucumber and Selenium. With its in-the-cloud, cross-browser capabilities, Sauce OnDemand speeds up the testing process and cuts out the need for expensive testing infrastructures, allowing Cucumber and Selenium to easily run in tandem.

Setting up Cucumber with Webrat and Selenium can be a dense process, but Sean documented it from start to finish, beginning with this blog post here. Watch the video from the live webinar, in which he demonstrated all the above and more.

Sean hopes to delve in to this topic even further at a later date, so don’t forget to check out our webinar page to see what’s coming up in the future!

In the meantime, if you’d like to start testing your apps in the cloud without all this complexity, check out a risk-free trial of Sauce OnDemand today.

  • Share/Bookmark

Setting up Cucumber + Webrat + Selenium

June 15th, 2010 by Sean Grove

There’s quite a bit of information out there on getting these disparate tools to work together, but a great deal of it is out of date. To clear things up a bit, I’ve documented all the gems and modifications necessary to get these pieces of open source software up and running together. As time goes on, I’ll be expanding this blog post with notes about the pitfalls and various platform issues that may be discovered (I’m looking at you, Snow Leopard), but this should get most people up and running right away.

This is all using a clean REE environment via the poorly named but wonderfully written rvm, or Ruby Version Manager.

Gems you’ll need:

gem install actionmailer actionpack activerecord activeresource activesupport builder cgi_multipart_eof_fix cucumber cucumber-rails daemons database_cleaner diff-lcs fastthread gem_plugin gherkin json json_pure mime-types mongrel net-ssh net-ssh-gateway nokogiri rack rack-test rails rake rdoc rest-client rspec rspec-rails Selenium selenium-client sqlite3-ruby term-ansicolor trollop webrat

Some of those are not strictly necessary, but simply nice to have, while others solved some unexpected problems with the bare necessities. I’ll prune this list as feedback comes in from people’s experiences.

Points to watch out for:
Nokogiri: This was easily the worst on my Snow Leopard machine. It relies on the native libxml2, which had problems with 32/64 bit compatibility. No matter what I tried, errors kept coming up. I had to clean everything out with my MacPorts installation and force a universal installation.
Webrat: The Selenium server jar that was included by default caused no end of headaches. I had to manually go in to the directory, remove the default server jar, and download the newest version from the seleniumhq download page.

Once those are set up, you should be able to use Cucumber, Webrat, and Selenium together without too much headache. Then you can refer to our webinar video (which will be posted to the blog later this week) to learn how to use Cucumber to easily run Webrat and Selenium.

  • Share/Bookmark

A bit of sugar and parallelism for Rails and RSpec

March 9th, 2010 by Sean Grove

Even though we focus very heavily on full-stack acceptance testing for the rails world, we know other forms of automated tests are critical as well. Our rails developers here make pretty heavy use of RSpec unit tests, and it’s nice to understand how to run those in parallel as well.

If you’re looking at how to setup a rails and selenium testing environment, check out our last post.

Parallelize the specs

We’ll use the excellent parallel_specs to beat a bit of parallelism into our specs. It prepares a separate database for each test environment, groups the specs to divide amongst processes, and then starts up a rails environment with a separate database for each group of processes.

I’ll paraphrase the installation instructions for convenience.

Install the required plug-in/gem:

sudo gem install parallel
script/plugin install git://github.com/grosser/parallel_specs.git

Here’s the semi-ingenious point – yaml can interpret ERB, so we can pass in an environment variable to the database.yml specifying at launch which database we want it to connect to.

Open config/database.yml and add the following:

test:
  adapter: sqlite3
  database: db/xxx_test<% ENV['TEST_ENV_NUMBER'] %>.sqlite3
  pool: 5
  timeout: 5000

(You can of course replace xxx_ with your project name)
So for example, to have our tests run against the xxx_test2 database, we would use:

export TEST_ENV_NUMBER=2; rake db:test:prepare

But it doesn’t make much sense to invoke it manually. That’s what plug-ins are for! Let’s go head and create/migrate a few test databases:

export TEST_ENV_NUMBER=0; rake db:test:create; rake db:test:migrate;
export TEST_ENV_NUMBER=1; rake db:test:create; rake db:test:migrate;
export TEST_ENV_NUMBER=2; rake db:test:create; rake db:test:migrate;

Great, now you’re able to run your non-Selenium tests in parallel!

But what about Selenium tests?

Stay tuned for our article on Sauce Labs’ SpecStorm plugin, that allows you to run your Selenium tests in true parallel fashion with Selenium Grid or our very own Sauce OnDemand service.

  • Share/Bookmark

Running Selenium RSpec tests for Rails 2.3.5

February 9th, 2010 by Sean Grove

This article will get you setup with the bare-minimum environment to run Selenium tests with RSpec, for automated, full-stack testing of Rails apps. Once that’s done, we’ll work on polishing it for a nicer experience. In this post you find amalgamations of mostly outdated articles from around the internet updated to work with a modern rails system.

It’s written on a clean environment courtesy of rvm, so you may have some of the gems already setup.

A new Rails app, RSpec, and rspec-rails

First off, we’ll start with a new rails project:

gem install rails --no-ri --no-rdoc
rails rspec_saucerc
gem install rspec
gem install rspec-rails

Edit config/environment.rb and add:

  config.gem "rspec", :lib => false, :version => ">= 1.2.9"
  config.gem "rspec-rails", :lib => false, :version => ">= 1.2.9"

This sets up all the required files for RSpec to get off the ground, but we need to integrate it into Rails:

script/generate rspec

That will add the rake tasks, create the appropriate directories, and basic files. Now let’s get Selenium up and running!

Selenium

We’ll need to create a separate selenium database. in config/database.yml, let’s add:

selenium:
  adapter: sqlite3
  database: db/selenium.sqlite3
  encoding: utf8
  timeout: 5000

To get Selenium running under our RSpec stories, we’ll need the Selenium gem (note the capital “S” — it’s case-sensitive). Let’s also make sure we have the sqlite3 gem installed, and then prepare an appropriate environment for our selenium tests:

gem install Selenium
gem install sqlite3-ruby
cp config/environments/test.rb config/environments/selenium.rb

Open config/environments/selenium.rb and remove the last line that reads:

config.gem 'rspec-rails', :version => '>= 1.3.2', :lib => false unless File.directory?(File.join(Rails.root, 'vendor/plugins/rspec-rails'))

Sauce RC and Selenium RC

Sauce RC will broker all communication between rails and any browsers you might be driving. Get it from our downloads page for Windows or Mac, or use Selenium RC for Linux, and start it up so we can run our tests.

Testables

We’ll need some fodder to test. Use the RSpec generator, then create and migrate the database:

script/generate rspec_scaffold person name:string age:string language:string
rake RAILS_ENV=selenium db:create
rake RAILS_ENV=selenium db:migrate

Startup our test server

Rails tests do not normally bind to a webserver, so there is no way for Selenium to access the frontend. As a temporary workaround, we’ll manually invoke a Rails server using the selenium environment:

script/server -e selenium

Our spec helper

We will be using the Selenium gem, so let’s add it to the spec helper:

gem "selenium-client"
require "selenium/client"
require "selenium/rspec/spec_helper"

Example story

We’ll gloss over which types of tests should include selenium front-end testing for now, and just say it belongs in integration tests. Let’s run an example story that will fail the first time through, and we’ll then fix it.

Put the following in spec/integration/people_spec.rb:

require 'spec_helper'
describe "People" do
  before(:all) do
    @verification_errors = []

    @browser = Selenium::Client::Driver.new(
      :host => "localhost",
      :port => 4444,
      :browser => "*firefox"
      :url => "http://localhost:3000",
      :timeout_in_second => 90)

    @browser.start
  end

  before(:each) do
    @browser.start_new_browser_session
  end

  append_after(:each) do
    @browser.close_current_browser_session
    @verification_errors.should == []
  end

  it "should create a new Person with valid input" do
    @browser.open "/people"
    @browser.click "link=New person"
    @browser.wait_for_page_to_load "2000"
    @browser.type "person_name", "Jason Huggins"
    @browser.type "person_age", "26"
    @browser.type "person_language", "Albanian"
    @browser.click "person_submit"
    @browser.wait_for_page_to_load "30000"
    @browser.is_alert_present.should be_true
  end
end

Let’s try it out:

rake spec:integration
F
1) 'People should create a new Person with valid input' FAILED
expected false to be true
./spec/integration/people_spec.rb:22:

Finished in 7.443904 seconds

1 example, 1 failure

It’s failing as we expected it to (there shouldn’t be a javascript alert on submit). Let’s change the last line:

    @browser.is_alert_present.should be_false

… and try our test again:

rake spec:integration
.

Finished in 7.079062 seconds

1 example, 0 failures

Looks good! We finally have a working rails project with rspec runners and Selenium.

Weaknesses

Although we’ve got Rails, RSpec, and Selenium all working together, the relationship is not harmonious.

Automated tests in rails are not meant to serve the outside world, and as such they don’t bind to a webserver/port. Tests are run within the same process, which makes them quite fast, but prevents us from using browser-based frontend tools such as Selenium. To get around that, we manually fired up a rails server instance with the selenium environment, but this is messy for a number of reasons:

  • We have to manually start/stop the test server that selenium wants to access
  • We have to manually create the selenium environment’s database
  • We have to manually migrate the selenium environment each time there’s a schema change
  • Worse, we have to manually reset the database after each run
  • Running Selenium tests in serial is slow

We‘ve developed a plugin to automate some of these issues, called SpecStorm. We’ll go over installing it to get the most out of your tests (including running them in parallel) in the next post.

Notes:

Selenium matchers are case-sensitive: @browser.click “link=New Person” won’t match a link with “New person”

  • Share/Bookmark