Virtru Increases Platform Coverage by 600% Using Sauce Labs

Scale testing to better serve new and existing customers

Providing customers with the ability to easily protect and share data securely is foundational to Virtru’s business, which means the company’s data protection solutions must integrate with third-party platforms and apps. Any changes by the third parties that it integrates with could cause Virtru’s software to break, causing a poor customer experience. Daniel Crowley, Director of Engineering at Virtru, explains, “The big differentiator between our encryption solution and the competition is the ease of use, which means having a platform that's always up and running.”

Virtru’s goal was to better serve its customers through continuous testing on beta and early-release versions to catch issues promptly, prevent software failure, and proactively identify and resolve production defects before customers encounter them. However, years ago when Virtru was a startup, they had limited resources, and the company struggled to identify issues in-house, often learning about problems from Customer Support. Virtru knew it had to improve user experience, which required more robust testing. 

Virtru needed a partner to provide a comprehensive and scalable testing platform so it could continue to focus on improving the customer experience. In the search for the right provider, the company wanted a solution that could simultaneously run tests in multiple environments and maintain reliability while its teams constantly added features. It also needed a partner that could support its move to mobile and growth into new markets with expanded platform solutions and third-party integrations.

Continuous Testing Throughout the Pipeline

Virtru selected Sauce Labs as the partner it needed. “We went to Sauce Labs because we were able to test on multiple platforms,” Crowley shares. “We could test continuously. It was a service that we could rely on and spin up at any time.” 

Now, Virtru has been a Sauce Labs customer for seven years. Initially, Virtru’s testing required 10 Sauce Labs virtual machines (VMs) running concurrently. As Virtru grew, the needs of the business evolved– and Sauce Labs scaled to support it. Today, Virtru has expanded its application to support mobile platforms, and has adopted BDD (Behavior Driven Development) methodologies to make testing accessible to the entire organization. Virtru can run tests with Sauce Labs on every step and every environment change that it has, from development to staging to production.

Concurrent Testing, Faster Releases and Product Growth

With Sauce Labs, Virtru was able to expand its coverage from 2 to 14 platforms—a 600% increase. Now, Virtru uses 50 Sauce Labs VMs to increase testing coverage, reduce test run times, help shorten the development lifecycle, and accelerate releases. “Growing as fast as we did, we needed Sauce Labs to make sure test coverage was where it needed to be,” explains Crowley.

Using Sauce Labs, Virtru can prevent issues by testing throughout its development pipeline. This includes quality checks as developers write code, which helps avoid regression bugs before deploying into production, as well as in its production environment, which quickly uncovers third-party changes. Testing concurrently results in faster release times, giving Virtru more confidence in its own solution and helping the company provide a better user experience. “Running tests all day on beta or early-release versions of the different platforms we integrate with allows us to better serve our customers,” Crowley states. 

Without Sauce Labs, Virtru wouldn’t be able to perform the level of testing required to ensure the best user experience. “Two or three years ago, we ran 100 tests per day. Now it's 6,000 tests per day,” Ken Ho, Staff Engineer at Virtru, shares. Using Sauce Labs has also allowed Virtru to increase development team productivity, allowing them to become more digitally confident. With Sauce Labs measuring defects throughout its development lifecycle, Virtru can quickly uncover issues and work on fixes immediately.

With the range of platforms and apps it integrates with, Virtru uses Sauce Labs to scale testing across the whole organization, and the company doesn’t require separate teams to maintain the different test environments it needs. The holistic offering from Sauce Labs gives Virtru digital confidence while saving time and resources. Crowley explains, “Our site reliability team can focus more on our critical services to our platform and make sure that those are constantly improving instead of having to upkeep VMs.”

Growing the business, and the partnership

As Virtru further expands its business, it plans to remain a loyal Sauce Labs customer. “The engagement of the Sauce Labs community has kept us very happy,” Crowley confides. “We’ve never even considered moving off of it. We’re upgrading our framework often because we have opportunities to make things better, and Sauce Labs has definitely been a partner with helping us to do that.”