As testers who focus primarily on testing web and desktop applications, what is the difference between regression testing traditional web/desktop applications and mobile applications?
The standard Wikipedia definition of regression testing is the process of testing changes to computer programs to make sure that the older programming still works with the new changes. Regression testing is a normal part of the program development process and, in larger companies, is done by code testing specialists.
The ultimate goal of regression testing is to ensure that new features or application updates do not introduce new bugs.
The difference between testing traditional web and desktop vs. mobile is really a matter of scope. The process is essentially the same, but the scope and development methodologies involved make mobile testing more complex.
For example, web apps consist of which browsers and browser versions will be supported (MSFT Edge, IE, Firefox, Chrome, Safari, etc.). This can be complex as well, testing for functionality, usability, security and performance.
Mobile testing consists of the various mobile browser combinations, but must consider native, hybrid, responsive and progressive web apps as well. Add to that the many device and device OS combinations of Android and iOS, and you have yourself quite a testing challenge.
Agile, DevOps & Continuous Testing
The speed of release and change makes mobile development agile. Mobile app development naturally follows the same paradigm as Agile and DevOps as it relates to continuous testing and continuous delivery.
Mobile requires that continuous testing (the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks) is a key component to the overall regression testing strategy. In order to properly build out a mobile regression testing strategy, it is imperative that the dev/test teams are well equipped with the following:
A comprehensive web and mobile testing platform - building a test lab from the ground up is timely and expensive. The best option is to use a cloud-based, device lab solution that provides an extensive choice of real devices, as well as emulators and simulators. This should also include parallel testing so that test execution can be done in a shorter amount of time.
Highly scalable, highly available solution - developers and testers need to ensure that the infrastructure for mobile testing allows the team to expand coverage as the test suite grows. The goal is to decrease test execution time while providing fast feedback, and to ensure that the team spends less time chasing false failures and more time on actual testing and defect remediation.
CI/CD optimization - in order for the regression testing efforts to keep up with the fast pace of continuous delivery, the process must have a tight integration with the development workflow and mobile app delivery pipeline. The goal is to eliminate any unnecessary manual steps, and promote an automation-first philosophy throughout the testing process.
Although mobile regression testing can be a challenge, the risks of flaky software can be reduced by making sure your organization has the right strategy in place. For more information on how to make the most of your mobile testing strategy, we recommend the following to help get you started:
Analyst Reports/Studies
Sauce Labs White Papers:
