Despite a strong background in automation, Lookout did little automated browser testing before turning to Sauce Labs. They opted to run their tests in the cloud rather than deal with the hassle and prohibitive cost of building and maintaining their own grid.
Using Sauce Connect to proxy traffic between their servers and Sauce, Lookout was able to add automated test coverage to their development process easily, with more capacity and flexibility than a local Selenium setup.
The Challenge
Lookout, the world’s leader in mobile security, creates products that are used by millions to protect individuals, businesses, governments, and critical infrastructure from the growing threats in the post-PC era.
Prior to December of 2011, the Lookout team had a strong background in building automated tests in Ruby, however they did not have extensive browser automation yet. As Lookout continued developing its products, it became clear their existing apps required more test coverage, so they began to convert existing test cases over to Selenium.
Lookout wanted to add more test automation to their process, however the Quality Engineering team at Lookout had experienced firsthand that building and maintaining their own Selenium Grid would not only be a hassle, but also prohibitively expensive. They needed a solution that would allow them to run tests more easily and reliably than running their own grid.
The Solution
Lookout turned to Sauce to run tests in the cloud using Sauce Connect, Sauce’s secure tunneling system. Members of the Lookout QE team had previous experience building and running Selenium Grids, and because they knew how unreliable and labor-intensive it was, decided never to run Selenium infrastructure internally.
Lookout runs Jasmine unit tests and Selenium tests before deployment to check for bugs and monitor the health of the build. They use Jenkins for CI, and pull Sauce test results into Jenkins. Like any mobile company, there are times when fixes need to be quickly implemented. At times Lookout aims to deploy within three hours, requiring that all testing is done on short notice. With Sauce, they can parallelize testing and increase capacity with the click of a button, allowing them to test code changes under tight deadlines.
The Results
With Sauce, Lookout has saved time and money running tests in the cloud. “Forced to build out and maintain our own infrastructure for cross-browser testing at scale, would have cost us valuable time testing the product.” said Dan Dover, Principal Quality Engineer at Lookout. "By outsourcing the in-house grid, Lookout has a more reliable and stable grid."
The ability to parallelize to run tests faster has also led to changes in the way Lookout approaches optimizing test suites. Because they can run many tests concurrently, they’re spending less time making the tests themselves run faster and more time adding additional parallel threads.
Developers and the QE team use Sauce’s visual assets, like screenshots, videos and logs to more quickly identify issues. Overall, using Sauce to run automated tests in the cloud allows developers at Lookout to easily reproduce issues and spend less time debugging, without the hassle of maintaining a costly local grid. Sauce enables an extremely reliable grid, and is flexible enough to scale instantly.
