Limited Testing Coverage and Poor User Experience
In the company’s early years, it focused on building out different software modules to create a comprehensive product, with quality assurance (QA) team members and manual testing siloed in each area. However, testing was only performed on the developers’ preferred browsers, allowing mistakes to occasionally slip through the cracks. Plus, the focus on manual testing wasn’t scalable to support the company’s growth and slowed validation times and the release of new features.
Autodesk needed a scalable solution to ensure digital confidence and know they were creating the best possible experience for their customers. The company decided to adopt agile methodology and shifted their focus toward automation. The team had three criteria for evaluating a testing tool: strong technical support, easy integration into its existing environment, and a single tool to cover all its functional and visual testing needs.
One Tool for a 360° View of Customer Experience
After extensive research, Autodesk became interested in Sauce Labs for its strong local support. It decided to perform a proof of concept (POC) with Sauce Labs to see how Sauce Labs functioned within the context of Autodesk’s code base.
The Autodesk POC made it clear that Sauce Labs was the right choice. “In the POC, we saw that the solution was extremely straightforward and very readable, and it combines an automation testing tool with a visual regression tool. We realized this would save QA time, make it easy to write and maintain tests, and help us debug any issues we uncover,” Daniel Gefen, Engineering Manager at Autodesk, explains.
“Another thing that differentiated Sauce Labs was seeing that it invests in its platform by developing it further,” adds Zohar Liran, Head of Data Intelligence Group at Autodesk. “When we were reviewing the platform, Sauce Labs kept adding features and platforms, even mobile testing. While not all these features were things that we would immediately use, it gave us a level of confidence knowing they invest in the future. We knew we were in good hands.”
Gaining a Competitive Advantage and Improving Customer Satisfaction with Faster Releases
Now, with Sauce Labs, Autodesk can discover errors earlier in the delivery pipeline. Visual testing is run early on UI components and later in end-to-end tests, so testing can be done at different points in product life cycles and by different teams. Automation has improved the productivity of teams while decreasing operating costs. For example, deployment cycles were reduced from weeks to hours, and, because Autodesk can catch issues earlier, they’re faster and less expensive to fix. “Before Sauce Labs, the process didn’t include covering edge cases of different platforms and browsers, and it was manual,” Liran explains. “With Sauce Labs, we have a single visual snapshot for visual and functional regressions, so less assertions are needed. This reduces the complexity of writing and maintaining automated tests. Now, teams move faster with confidence and we can speed up releasing new features to customers, giving us a competitive advantage.”
Automated testing has also increased Autodesk’s speed to market as the company scales. Before Sauce Labs, Autodesk released weekly or bi-weekly, but with this new confidence in its release process, it has continuous deployment and can release anytime. Now, 100 tests run in parallel every time there’s a change to the product. “With Sauce Labs, we have confidence on three key levels,” Liran says. “Contributors releasing the feature know it was tested automatically on multiple browsers and works. Managers know that nothing will break when teams release new features or updates. And product owners know they can deliver better products to market, faster.”
Gefen adds, “It not only helps us deliver a better customer experience in the short term, but it will also ensure a great experience in the long run because once you write the test, the test will always run.” Sauce Visual (formerly Screener) and visual regression testing have had a particularly positive impact on customer experience. Before, Autodesk could not test the visual aspects of its UI with traditional automation. Because UI bugs were not caught, user experience suffered. Now they can automate this testing with Sauce Visual, dramatically increasing their test coverage. The improvement in customer experience is reflected in the company’s growth of monthly active users (MAUs). For a recent project, Autodesk already has 100,000 MAUs.
Scaling with a Reliable Testing Partner
As Autodesk scales and extends its offerings, it will continue relying on Sauce Labs. “Our plan is to increase our test coverage,” Gefen explains. “We want to expand our scale more with Sauce Labs to increase test, browser, and resolution coverage.” Autodesk also plans to add tablet and mobile testing with Sauce Labs within the next year.