The Challenge
Quick Base, Inc. provides a low-code application development platform. Their solution allows non-technical developers to create secure cloud applications without compromising IT governance and control. With a development platform as its core business, Quick Base simply couldn't afford to be distracted by managing and maintaining infrastructure and frameworks when that can be handled by another company. “It took a long time to get results because tests were fragile. We weren't following consistent practices,” says Karen Motyka, the Quick Base Director of Quality Engineering.
Running its UI regression suite overnight took about five hours, which prevented fast feedback loops and slowed down release times. To execute these tests, Quick Base used a local virtual machine running Chrome and Firefox. However, this didn’t provide the cross-browser coverage that the company needed, because most of its customers used Internet Explorer. In addition, every morning the QA team had to manually review the results of their regression suite and piece together what caused each defect.
The entire process was slow and inefficient, and staff lacked confidence in the results. “People didn't really pay attention to the automation because they just didn't believe that it was providing anything of value,” explains Motyka, “And that's the thinking we really needed to change in order to be successful.”
Quick Base wanted to stop UI tests from being a bottleneck. As a product development
organization, it needed stronger, consistent testing across all five primary browsers. It sought a solution to help enable and empower developers to test their code early and often across a variety of environments. “We have been very focused on speed and quality,” says Motyka “We want to deliver many more features and capabilities to our customers and do that quickly, without sacrificing quality.”
The Solution
Quick Base decided to provide teams with a fast, strong, and consistent way to test and, to that end, turned to Sauce Labs, a cloud-based automated test execution platform.
Starting with a free Sauce Labs trial, Quick Base found the solution easy to integrate, and a
conversation with Sauce Labs sealed the deal. “After talking with Sauce Labs,” says Motyka, “and seeing the support, parallelization, and partnership that we can have, and the ability to discuss ways in which we can improve—it was like, ‘Okay, this seems like a no-brainer.’”
Quick Base currently uses a mix of Selenium and WedDriverIO to run over 2,000 UI tests per day, across all five primary browsers.
The Results
Quick Base takes advantage of the Sauce Labs Continuous Testing Cloud to run hundreds of consistent and reliable tests each day in parallel, across browser platforms. “We've run almost 12,000 tests in the past seven days,” says Ken LaBak, a Senior Software Engineer in Test on Motyka’s team. “We can run those across all the different browsers. It takes about 15 minutes per check-in, which is a huge improvement versus the five-hour regression suite that we used to have,” says LaBak.
With Sauce Labs, Quick Base removed the bottleneck from its testing environment and gained confidence in automation. Now, the company can get features to market faster and with fewer defects. “We find issues early in the pipeline because we can run tests in parallel,” explains LaBak, “so the teams get feedback much more quickly, and we can address issues before they ever get into our production environment.”
Sauce Labs has delivered speed, parallelization, and browser coverage to testing, helping Quick Base offer higher quality features to customers, faster. “I don't think we'd be able to do this without Sauce Labs,” says LaBak “We wouldn't be able to validate the 35 features we now have in our new UI, running all these different types of tests.”
Recently, Quick Base scored a perfect 100 percent on Sauce Labs’ Continuous Testing
Benchmark (CTB), which shows the health of a continuous testing practice. Reports such as the CTB have been an unexpected but much appreciated benefit. “I needed a way to show the progress we are making with our test automation and the value this provides our organization. The CTB was a perfect mechanism for me to do that” says Motyka. “The CTB along with other Sauce Lab analytics also helps us identify opportunities for improvement so we can continually evolve our practices.”
Future: Improving the Testing Environment
Moving forward, Quick Base will continue its strong partnership with Sauce Labs, relying on its experts to keep advising Quick Base developers on best practices and continuing to play a critical role supporting Quick Base’s quality engineering. Quick Base has started several proofs of concept with Sauce Labs, including Sauce Visual, front-end performance, and Jenkins infrastructure plugins. The Quick Base team anticipates Sauce Visual helping significantly by safeguarding and catching regressions as it adds more engineers and performs more UI development.
