After being brought on to Campus Explorer, Sage introduced the team to Sauce Labs.
By outsourcing their Selenium Grid and VMs, Campus Explorer reduced the amount of time it took to test from 72 hours to 72 minutes. Paired with other time saving features such as team collaboration, continuous testing, and videos from the Sauce dashboard, Campus Explorer increased the quality of employee lives as well as the quality of the site.
The Challenge
Campus Explorer makes college and career search easy and fun through a comprehensive search and matching engine that boasts over 8,500 options.
Formerly a QTP user during his time at another company, Sage Rimal, a Senior QA Engineer at Campus Explorer, knew there was a better, comprehensive, and holistic way to approach Quality Assurance.
"QTP is expensive," Sage says. "It uses VBScript as a scripting language, and it only supports limited browsers - for instance, it doesn't support Mac or Safari in full mode, just beta."
In 2009, Sage discovered Selenium and moved over because it was open source, supported all platforms, and all major languages. He also appreciated the large community and amount of support for Selenium.
When he was brought on to Campus Explorer, he knew he wanted to work with Selenium, but acknowledged it was difficult to maintain a Selenium Grid and several VMs. "Maintaining the test environment was a full time job," says Sage. "On some occasions, I spent more time setting up the test environment than actually testing." Further, without an automated process in place, he found that manual testing their site took up to 3 days.
The Solution
Needing a better way to test, Sage came across Sauce Labs. "I moved to Sauce because of the wide array of browser and platform support. It’s easy to use, and easier to maintain," says Sage.
Sage’s team of 5 now runs over 70 tests in parallel 6 or 7 times a day as part of their Jenkins CI builds.
The QA department enjoys the easy collaboration features. "The entire team can look at a bug or error though screen-share," Sage says. "One of the biggest advantages that I have seen with Sauce is that I can record test failures, and watch the complete scenario playback. I think this is more useful than looking at the stack trace of errors."
The Results
The biggest value gain for Campus Explorer has been the ability to test on multiple browsers using one test script without having to do any internal setup. That, paired with the easy automation setup and browser validation in multiple environments, has been immensely helpful in speeding up the testing process. "With Sauce, I reduced testing time from 72 hours to 72 minutes," Sage says; a time savings of 98%. Prior to using Sauce’s video playback tool, debugging took an average of 40 minutes. Now the team regularly decreases time spent on each test failure by 75%, resulting in a much higher ROI on developer productivity.
But the time and cost savings aren’t the only benefits for Sage or his team. "Quality means a lot for me and not just in work life, it's also important in my personal life. In QA automation, the fear 'what if something's broken?' is always there," says Sage. Since using Sauce Labs, Sage no longer has work-related stress. "We have CI servers that run tests automatically and we get results immediately. Once everything passes, I don't have to worry about it anymore. I can just go."
