The Challenge
When real estate brokers want to help clients find their dream homes, they increasingly turn to BoomTown for help. A fast-growing real estate sales and marketing platform, BoomTown includes websites with the latest real estate listings and housing data, a customer relationship management (CRM) solution, and analytical tools.
BoomTown relies heavily on testing to deliver a solid experience to the thousands of brokers and home buyers that use its products every day. “We know that an exceptional experience will create raving customers, so we’re on a never-ending quest to provide the highest quality service to our users,” says Brian Baumgartner, Quality Assurance Manager, BoomTown. The company’s quality assurance team, though, primarily used manual, time-consuming testing processes. “We wanted to automate some of our more repetitive tests to free up QA analyst time,” says Joey Gryder, Senior Software Engineer in Test, BoomTown. However, BoomTown did not want to be responsible for its own onsite automation lab. “It’s a lot of work to have an automation lab,” Gryder says. “You need full-time engineers to maintain it. That’s not what we want to do here. We decided we needed a cloud-based testing solution instead.”
The Solution
In late 2014, after considering BrowserStack and other testing platforms, BoomTown chose Sauce Labs, a cloud-based automated testing solution. "We found Sauce much easier to get up and running, and it was also more stable and reliable than the other solutions," says Gryder. Additionally, Sauce Labs supports the Selenium open-source testing framework, which BoomTown had been using.
The nine-person BoomTown QA team currently uses Selenium to run automated testing of its websites and CRM solution against multiple desktop browsers with Sauce Labs. The team runs 100 nightly tests on at least two browsers per build. Tests include end-to-end functionality tests of various end-user scenarios as well as regression tests to catch bugs.
Using Sauce Labs virtual machines, BoomTown runs eight tests in parallel each night for regression runs. "Early on, we knew that parallel testing was important, because we want quick feedback on builds," says Baumgartner. "With parallelization in Sauce Labs, we make sure our tests run in complete isolation from each other."
The Results
With Sauce Labs, BoomTown is able to optimize its continuous integration and delivery processes by enhancing its automation workflow and reducing test times. For example, the company has reduced its longest test runtime from more than one hour to just minutes. “In one of our longest tests, which is our CRM back-end test, we use eight parallel tests on Sauce Labs and the total runtime is 13 minutes,” Gryder says.
“Previously, it would have taken more than 90 minutes to complete the entire run.”
BoomTown also saves time by taking advantage of built-in functionality in Sauce Labs. “We get a lot of things for free with Sauce Labs that we would have had to build into our automation framework otherwise,” says Gryder. “For instance, we use the test dashboard to investigate run results, and we use video playback to save time on troubleshooting.” As a result, BoomTown does not need to manage an internal testing infrastructure. “With Sauce Labs, we eliminate the need to build and maintain a costly in-house lab. We can offload the testing infrastructure to Sauce Labs so we can focus our time on expanding our testing,” says Baumgartner. In fact, by using Sauce Labs, BoomTown avoids having to employ an additional QA employee. “If we were supporting our own in-house automation lab, we would probably have to have another full-time engineer on staff,” says Gryder. “By using Sauce Labs, we eliminate that need.”
In the near future, BoomTown plans to support its mobile testing efforts on Sauce Labs, using the Appium framework to run automated tests against mobile browsers. “With Sauce Labs, we know we’ll be able to continue releasing high-quality software for the real estate industry,” says Baumgartner.
