Massive Reduction of Selenium Costs with Sauce Labs

Posted Oct 4th, 2011

This post was originally written for Okta's blog by QA Lead Denali Lumma. Okta is an identity & access management company for cloud/SaaS applications. It has been reprinted with her permission. At Okta we provide enterprise grade identity management from the cloud.  Specifically, Okta delivers single sign-on across all your web applications, centralize user management and control, and integration with Active Directory.  We invest heavily in test automation and validation.  A lot of great testing tools are utilized, including Selenium.  We currently have about 1000 Selenium tests in our production test system.   These tests are deployed and run in the cloud with our trusted partner; Sauce Labs.  Currently, Selenium tests run continuously on check-in across master and release branches against our supported browser/ version/ OS matrix.  This matrix currently includes Firefox 3.6 on Linux, Firefox 7 (latest stable version) on Windows, Internet Explorer 8 on Windows, Internet Explorer 9 on Windows, Google Chrome on Windows, Safari 4 on Windows and Safari 5 on OSX.  (Safari 5 on OSX is run locally until the up and coming Mac VM feature is released by Sauce Labs.) The decision to use Sauce Labs as the Selenium client rather than running tests internally on a roll your own Selenium Grid configuration was methodically considered by our engineering team. Even before we began to consider working with Sauce Labs, internally a general question would come up every now and again: why bother with Selenium?  The answer was simple: we have no choice. Because of our business focus, we must follow industry best practices with test automation across all levels of the stack including the GUI in order to ensure excellent product software quality.  Yet, there is no escaping that web application test automation with Selenium is expensive.  Engineers at our company report spending anywhere from 10 – 60% of their time writing, reviewing and maintaining Selenium tests.  But the cost of not implementing Selenium testing is beyond high—it’s a game-ender.  Without constant deep and broad test feedback developers are unable to introduce changes safely and responsibly at the constantly accelerating pace achieved by the world’s best software companies—our competitors and your competitors.  But we’ve managed to reduce these costs significantly with Sauce Labs.  What we’ve found through our trial investigation and now full deployment is that Sauce Labs saves us tremendous time, and thus tremendous money. Read the rest of this post on Okta's blog

Written by

Ashley Wilson


SeleniumCross-browser testing