The company uses Sauce Labs to ensure high-quality software and cut regression test times from three to four days down to one.
eHarmony runs more than 1,000 tests every day on multiple operating systems and browsers as well as mobile emulators and simulators.
The Challenge
eHarmony is a leading online dating website based in Los Angeles, California. The popular site has grown to about 33 million members since its launch in 2000. Each day, an average of 438 single people marry someone they found on eHarmony.
For the eHarmony technical team, nothing is more important than the customer experience. “We want our customers to have a great experience with our company, and that means providing bug-free software and limiting the number of crashes on our apps,” says Amit Sainani, QA Manager at eHarmony. However, as the company rolled out an increasing number of new features and functionality to its applications, software testing cycles took longer and longer to complete. “We were doing some automated testing using an in-house grid, but it still wasn’t fast enough, and it didn’t provide the amount of coverage or control we needed,” Sainani says. To solve its challenges, the company wanted to move to a cloud-based testing solution. “We really needed to automate and run across a wider range of tests for different browser platforms,” explains Sainani.
The Solution
eHarmony decided to accelerate its testing and increase browser coverage by moving to Sauce Labs, a cloud-based automated testing solution. “I used Sauce Labs extensively in the past and had a great experience with it,” says Sainani. “It offers the kind of reliability, scalability and speed we need to obtain a quick feedback loop for quality analysis.”
The organization takes advantage of Sauce Labs virtual machines in parallel to run more than 1,000 daily tests across two QA environments. eHarmony ‘s QA team relies on Sauce Labs mobile emulators and simulators to automate testing of native and mobile web apps. The company runs automation tests on desktop apps on the Windows and Mac OS platforms across Safari, Firefox, Edge and Chrome browsers. Sainani says, “The emulators are easy to set up, and they’re working out really well. They help us see if new code has any changes on regression tests.”
eHarmony currently uses Sauce Labs to run tests on seven different platform combinations, and the company’s front-end SDETs rely on the Sauce plugin for Jenkins to manage all testing platform parameters. Additionally, eHarmony uses Sauce Connect™, a proxy server that opens a secure connection between a Sauce virtual machine and an application eHarmony wants to test that is behind the company’s firewall.
The Results
eHarmony has significantly reduced its software testing time by using Sauce Labs for automated tests. “For a large software release that requires testing many different areas of an application, it used to routinely take three to four days to run a full regression,” states Sainani. “We have reduced that to just one day using Sauce Labs, because we can automatically run so many different tests across different browser platforms.”
The company performs multiple software releases every day, and the team is confident in the quality of each release. “Using Sauce Labs, we can catch defects sooner in the process, before they go into production and impact our customers,” says Sainani. “When we come into the office in the morning and see that failures are low from overnight tests, we have peace of mind. It gives us confidence that the code at that moment is in great shape, ensuring higher-quality software. Ultimately, we can get code to market faster and with fewer defects.”
eHarmony also no longer needs to spend time managing its internal test grid. “We don’t have to mess with managing the test grid, because everything is set up in the cloud on Sauce Labs,” remarks Sainani. “We don’t even think about it, and we get much more efficiency in our testing.”
The company expects Sauce Labs to continue to play a critical role in its testing environment. “We’re really big on automation here, and I’m proud of what our team has accomplished over the years,” Sainani says. “We have a robust test environment, and Sauce Labs is a great tool for us to have when it comes to supporting that environment and delivering high-quality software.”
