Posts Tagged ‘selenium’

Dzone Covers Monocle and Async Programming

July 29th, 2010 by Jeff Goldsmith

Mitchell Pronschinske reports on 2010/07/28 – Greg and Steven Hazel from Sauce Labs have recently built what they call, “An async programming framework with a blocking look-alike syntax”. This framework, named Monocle, is focused on being portable between event-driven I/O frameworks. Currently, Monocle supports the Twisted and Tornado frameworks.

For those who haven’t heard of Sauce Labs, they were co-founded by John Huggins, the creator of Selenium. Sauce Labs’ free and commercial tools build on top of the core Selenium testing framework.

The emergence of Monocle could indicate that Sauce Labs is taking a more focused look at event-driven code and its role in concurrent web performance. Event-driven code is efficient and intuitive, but sometimes procedures are are split up and code is expanded in a not-so-good way.

Read more at Dzone

  • Share/Bookmark

Trends in Testing: Continuous Integration, Mobility, Open Source, Cloud

May 10th, 2010 by Zack Samocha

Functional testing is the automation of web app testing across several platforms or browsers. In the past, functional testing was less common due to long development cycles and the lack of multiple browser options. It was done primarily in-house via proprietary software like Rational or QuickTestPro.

Today, there are four trends converging to radically change how organizations test web applications:

  1. Continuous integration – An increased emphasis on performance of the software through the end customer’s perspective is leading agile development. Gartner predicts that by 2012 agile development methodologies will be used by 80 percent of all software development projects. Teams are shifting away from rigid quality control to quality that is demonstrable to the end user. With this shift, comes a need for quick, simple and automated testing tools.
  2. Open source – Open source tools, like Selenium, are viable for functional testing. Today’s development environment has dramatically changed and a lot more people are contributing than ever before. The major advantages of open-source are speed, time to market, time to value, and the ability to reach and grow a developer and user community. In fact, Gartner predicts that by 2012, 80 percent of all commercial software will include elements of open source technology.
  3. Cloud computing – Testing in the cloud is an affordable and scalable alternative to testing behind a firewall. It is expensive and time consuming to maintain on site test infrastructures that cover a vast number of browsers (and versions) of operating systems in several languages. The increase in the adoption of cloud computing creates an opportunity to leverage the space for functional testing. We believe the life cycle for cloud-based applications will look different in the future. New solutions that are cloud based will support cloud-base applications.
  4. Mobility – Websites need to support multiple browsers such as Internet Explorer, Firefox, Chrome, Rockmelt, Opera on various platforms including Windows and Mac. Smartphones only add to this “Browser War 2.0” battle with the plethora of mobile operating systems. These various environments make the infrastructure required more complex and the need for functional testing more relevant that ever before.

Are there other trends beyond continuous integration, mobility, open source and the cloud that will change how organizations test their web applications?

  • Share/Bookmark

Selenium participating in Google Summer of Code (GSoC)

April 6th, 2010 by John Dunham

Selenium has been accepted into the Google Summer of Code program.  This means participating students will get $5000 USD to hack on Selenium-related Open Source projects in the summer, while getting mentored by contributors of the Selenium project.   The deadline for application is April 9, 2010 and Selenium still has open slots -  from writing python bindings for Selenium 2.0 to writing mobile support for Android and iPhone to Selenium IDE 2.0.  You can suggest projects as well.  Once you have submitted your application, you will have until April 18, 2010 to fine tune your proposal with our mentors.  Google will announce the list of accepted students / proposal by April 26, 2010.

You can find more information here:
http://code.google.com/p/selenium/wiki/GoogleSummerOfCode

Link to apply:
http://socghop.appspot.com/gsoc/student/apply/google/gsoc2010

If you know students (or people that have children :-) attending college, please pass this info along.

This is a great opportunity for students to work on open source projects that’s used by millions of developers and QA professionals, gain skills in Selenium and ultimately help drive higher quality software everywhere.  Do note that most of the work will be done off-site, however, if you are paired up with mentors in San Francisco, LA, Portland, London…  you may have a chance to work with them directly too.

  • Share/Bookmark

Selenium Tips: Parametrizing Selenese tests

March 12th, 2010 by Santiago Suarez Ordoñez

Even though I believe Selenium IDE should just be a transition environment for new Selenium users to reach Selenium RC, we know a lot of our users keep all their tests in Selenese and run them using Selenium IDE as their main testing tool.

That’s why at Sauce we have developed Sauce IDE, our own extension of Selenium IDE that combines Selenium RC’s cross-browser capabilities with the simplicity and interactive design of Selenium IDE.

To continue empowering the IDE users, I’m writing this tip of the week that will help making Selenese tests easier to maintain and write.

As you all know, Selenese is not a real programming language. Variable storage in it is pretty rough and there isn’t an easy way to share information between different Test Cases in the same Test Suite.

However there is a way to do it, through the use of the user-extensions.js file. Let me show you how in two easy steps:

First, create a user-extensions.js file with the following content:

storedVars["url"] = "/staging/search";
storedVars["title"] = "Search Your Item";
storedVars["search-string"] = "234234kjkj";

Then open Selenium IDE, or even better Sauce IDE, and in the Options menu, select the file in the “Selenium Core Extensions” field.
Close and open Selenium IDE again and you should be ready to use that variable in any test that you write. Even better, throughout your whole test suite.

For example:

open ${url}
assertText h1 ${title}
type search-field ${search-string}
click btnSearch
assertTextPresent Search Results for ${search-string}

This may not look super useful at first, but once your test suite grows, keeping things that change from time to time in a single place, makes it really easy to maintain, so if your application’s URL changes from “/staging/search” to “/search”, you will only need to fix the user-extensions.js file and all the hundreds of tests you could have will pass the same way as they did before.

Check the wikipedia’s page on DRY for more information about this programming technique.

Hope you found this useful. If you did, please let us know in the comments.

Santi

  • Share/Bookmark

Selenium Tips: CSS Selectors in Selenium Demystified

January 29th, 2010 by Santiago Suarez Ordoñez

Following my previous TOTW about improving your locators, this blog post will show you some advanced CSS rules and pseudo-classes that will help you move your XPATH locators to CSS, a native approach on all browsers.

Next sibling

Our first example is useful for navigating lists of elements, such as forms or ul items. The next sibling will tell selenium to find the next adjacent element on the page that’s inside the same parent. Let’s show an example using a form to select the field after username.

</input> </input>

Let’s write a css selector that will choose the input field after “username”. This will select the “alias” input, or will select a different element if the form is reordered.

css=form input.username + input

Attribute values

If you don’t care about the ordering of child elements, you can use an attribute selector in selenium to choose elements based on any attribute value. A good example would be choosing the ‘username’ element of the form without adding a class.

</input> </input> </input> </input>

We can easily select the username element without adding a class or an id to the element.

css=form input[name='username']

We can even chain filters to be more specific with our selections.

css=input[name='continue'][type='button']

Here Selenium will act on the input field with name=”continue” and type=”button”

Choosing a specific match

CSS selectors in Selenium allow us to navigate lists with more finess that the above methods. If we have a ul and we want to select its fourth li element without regard to any other elements, we should use nth-child or nth-of-type.

    <p>Heading</p>
  • Cat
  • Dog
  • Car
  • Goat

If we want to select the fourth li element (Goat) in this list, we can use the nth-of-type, which will find the fourth li in the list.

css=ul#recordlist li:nth-of-type(4)

On the other hand, if we want to get the fourth element only if it is a li element, we can use a filtered nth-child which will select (Car) in this case.

css=ul#recordlist li:nth-child(4)

Note, if you don’t specify a child type for nth-child it will allow you to select the fourth child without regard to type. This may be useful in testing css layout in selenium.

css=ul#recordlist *:nth-child(4)

Sub-string matches

CSS in Selenium has an interesting feature of allowing partial string matches using ^=, $=, or *=. I’ll define them, then show an example of each:

^= Match a prefix
$= Match a suffix
*= Match a substring
css=a[id^='id_prefix_']

A link with an “id” that starts with the text “id_prefix_”

css=a[id$='_id_sufix']

A link with an “id” that ends with the text “_id_sufix”

css=a[id*='id_pattern']

A link with an “id” that contains the text “id_pattern”

Matching by inner text

And last, one of the more useful pseudo-classes, :contains() will match elements with the desired text block:

css=a:contains('Log Out')

This will find the log out button on your page no matter where it’s located. This is by far my favorite CSS selector and I find it greatly simplifies a lot of my test code.

Tune in next week for more Selenium Tips from Sauce Labs.

Sauce Labs created Sauce OnDemand, a Selenium-based testing service that allows you to test across multiple browsers in the cloud. With Selenium IDE and Selenium RC compatibilities, you can get complete cross-platform browser testing today.

  • Share/Bookmark

Selenium Tips: HTTPS and self-signed certificate exceptions

January 22nd, 2010 by Santiago Suarez Ordoñez

This week in our support email, help@saucelabs.com, we’ve seen a considerable number of cases of problems with Firefox and the Invalid Certificate warning thrown when a development environment is using the production’s certificate for HTTPS URLs, which causes the browser to wonder about the website’s identity.

Invalid Certificate warning

Warning displayed by Firefox 3.5 in this situation

If you do much browser-based testing, you have surely dealt with this situation more than once. In the manual testing world, you just tell your browser to add an exception for that certificate and ignore further errors, and you can forget about seeing that annoying warning anymore.

But when you move on to automation in the Selenium world, things are not always that simple. In most browsers, adding a certificate exception will work, as Selenium shares the same session as the user and will find the exception as you do. In Firefox, though, Selenium RC creates a special profile each time the browser is started and there’s no trail of the user settings in it.

The workaround for this problem is to create your own Firefox profile, with the specific certificate added on it by hand, and then tell Selenium to launch the browser based on that profile.

Another interesting approach, the one we take at Sauce Labs, where we can’t do this kind of trick, because we just don’t know which certificate the user will need before their test starts, is the use of RCE (Remember Certificate Exception), which is  a plugin that will automatically detect the warning and make the browser go through it, returning the control to Selenium after 4 or 5 seconds. Notice that if you use this approach, you will need to make sure your tests will tolerate this 5 secs additional delay to open the page.

Note: We currently support RCE in our Firefox 3.0 machines, and will be porting this extension to Firefox 3.5 soon.

You can find more info about RCE on its plugin page or in the author’s blog post.

Update: Adam Goucher, one of the big minds in the testing world just wrote a blog post that we couldn’t have written better ourselves about the first and most important advice regarding HTTPS and testing: Do yourself a favour and don’t test using HTTPS

  • Share/Bookmark

Running your Selenium tests in parallel: Clojure

December 28th, 2009 by Miki Tebeka

Clojure is a dialect of Lisp that runs on the JVM. It has very elegant syntax and a novel approach to concurrency.

In this installment, we’ll build a Clojure framework for running Selenium tests in parallel. We’ll use Clojure’s Java interop to interact with Selenium, and agents to distribute the work. Once you have the tests executing in parallel, you can run them against your internal selenium farm or use our Sauce OnDemand cloud hosted service to get running with no further configuration.

First, the framework, in about 50 lines of code:

(import '[com.thoughtworks.selenium DefaultSelenium])

(defn- run-with-client
  "Run a test function with new client"
  [test-fn opts]
  (let [client (new DefaultSelenium (:host opts) (:port opts) (:command opts)
                    (:url opts))]
    (.start client)
    (try
      (test-fn client)
      (catch Exception e false)
      (finally
        (.stop client)))))

(defn- run-single-test
  "Run a single test, append test result to results"
  [test results opts]
  (let [test-fn (test :test)
        value (run-with-client test-fn opts)
        res {:test (test :name) :value value}]
    (dosync (alter results (fn [old] (conj old res))))))

(defn- gen-agents
  [num-agents]
  (take num-agents (map agent (repeat nil)))) 

(defn- parse-options
  "Get options as hash map, provide defaults"
  [args]
  (let [opts (apply hash-map args)] ; Convert [:port 4444] to {:port 4444}
    (-> opts
      (assoc :port (:port opts 4444))
      (assoc :host (:host opts "localhost"))
      (assoc :command (:command opts "*firefox"))
      (assoc :url (:url opts "http://localhost/"))
      (assoc :num-agents (:num-agents opts 4)))))

(defn run-tests
  "Run tests in parallel, return list of test results.
  After tests are done, passes the list of results to reporter.

  A test has the format
  {
    :name NAME
    :test (fn [client] ...)
  }
  "
  [tests reporter & args]
  (let [opts (parse-options args)
        agents (gen-agents (:num-agents opts))
        results (ref [])]
    (doseq [[test agent] (map vector tests (cycle agents))]
      (send agent (fn [_] (run-single-test test results opts))))
    (doseq [agent agents] (await agent)) ; Wait for tests to finish
    (shutdown-agents)
    (reporter @results)
    @results))

(defn tests-failed?
  "Check if all tests passed"
  [results]
  (not (empty? (filter false? (map :value results)))))

Now let’s define some tests and run them:

(require 'selenium)

(def test-google
  {
   :name "google"
   :test (fn [client]
           (doto client
             (.open "http://www.google.com")
             (.type "q" "Sauce Labs")
             (.click "btnG")
             (.waitForPageToLoad "5000"))
           (.isTextPresent client "Selenium"))
   })

(def test-yahoo
  {
   :name "yahoo"
   :test (fn [client]
           (doto client
             (.open "http://yahoo.com")
             (.type "p" "Sauce Labs")
             (.click "search-submit")
             (.waitForPageToLoad "5000"))
           (.isTextPresent client "Selenium"))
   })

(def test-bing
  {
   :name "bing"
   :test (fn [client]
           (doto client
             (.open "http://www.bing.com")
             (.type "q" "Sauce Labs")
             (.click "go")
             (.waitForPageToLoad "5000"))
           (.isTextPresent client "Selenium"))
   })

(def tests [test-google test-yahoo test-bing])

; Our fancy reporter :)
(defn reporter
  [results]
  (doseq [r results]
    (println r)))

(let [results (run-tests tests reporter)]
  (if (tests-failed? results)
    (System/exit 1)))

To run the system locally, you’ll need the Selenium server and client jars and the Clojure jar (we’re using 1.1.0). You can use Sauce RC to get a Selenium server up and running using a GUI-based installer. Then, just issue the following command:

java -jar selenium-server.jar&
java -cp selenium-java-client-driver.jar:clojure.jar \
    clojure.main test-sauce.clj

(See the run-tests script, you can control the number of concurrent tests by providing :num-agents N to run-tests)

To run it against our cloud hosted service without having to configure any servers, simply follow the instructions at Sauce OnDemand documentation and supply the right options to run-tests. It will probably look something like:

(run-tests tests reporter
           :host "saucelabs.com"
           :command "{
              \"username\" : \"SAUCE-USER-NAME\",
              \"access-key\" : \"SAUCE-API-KEY\",
              \"os\" : \"Windows 2003\",
              \"browser\" : \"firefox\",
              \"browser-version\" : \"3.5.\"}"
           :url "http://saucelabs.com")

You won’t need to run any servers locally, just execute the clojure test:

java -cp selenium-java-client-driver.jar:clojure.jar \
    clojure.main test-sauce.clj

Future enhancements to this framework might include better reporting, and integration with the clojure.test testing framework.

All of the code from this installment, including the needed jar files can be found at http://github.com/saucelabs/parallel-test-examples/tree/master/clojure.

  • Share/Bookmark

Creating Selenium Tests for Mature Ruby on Rails Project

December 24th, 2009 by John Dunham

Consultant Sarah Mei talks about effectively using outsourcing to ‘catch up’ implementing Selenium tests on an existing Ruby on Rails project.

  • Share/Bookmark

Top 10 Reasons to Use Selenium

December 21st, 2009 by John Dunham

Last week we hosted the December edition of the San Francisco Selenium Meetup. Chosen topic of the evening was “Selenium-related lightning talks” with some delightful results. Amit Kumar of Betable presents “Top 10 Reasons to Use Selenium.” Enjoy!

  • Share/Bookmark

Selenium users talk about their experience with Selenium

December 21st, 2009 by John Dunham

Users talk about their favorite Selenium success stories and advice to new users at the 12/15/09 SF Selenium Meetup.

  • Share/Bookmark