How can you future-proof your mobile testing strategy? Given the rate at which significant mobile hardware and software updates appear, this is a critical question to answer.
Keep reading for tips on ensuring that your mobile testing workflow is ready to keep evolving, along with the mobile hardware and software it needs to support.
The ever-changing nature of mobile testing
One of the challenges of mobile software testing is that mobile devices evolve and change much more quickly than PCs. Sure, PC vendors might roll out new versions of their devices that contain new hardware updates on a regular basis. But from a software tester’s perspective, those updates don’t usually matter. They don’t tend to include dramatic changes to things like screen size, CPU power, memory, or other items that can significantly impact application performance.
And as far as PC software environments go, major operating system updates typically take years to appear. Windows desktops have seen only three major new operating system releases (Windows 7, Windows 8 and Windows 10) in the last decade.
In the mobile testing world, things are trickier. New mobile hardware devices roll out all the time. In 2017, Android Authority claimed the year to be one that we would all remember due to quality and yes, quantity. Ben Sin at Forbes released his favorite smartphone list of over 30 units that year - and that large figure doesn’t reflect other types of mobile devices (like tablets and wearables) that appeared during that time. Those devices can introduce major new hardware features, such as new screen constraints, battery lifetimes or even novel biometric features, all of which could become things that mobile software testers need to take into account.
From a mobile operating system perspective, mobile devices also change faster. For the last several years, a major new version of Android has appeared every year. iOS has been updated almost as frequently. With new mobile operating systems come new software environment variables that mobile device testing has to support.
Eventually, the pace of change in the mobile world may slow down. (There was a time when PC hardware and software evolved rapidly, too, but it ended at least a decade ago; mobile may go the same way sooner or later.) But for the foreseeable future, mobile devices will continue to evolve quickly. Mobile testing strategies that can’t accommodate future updates will leave organizations with gaps in their test coverage and prevent them from being able to deploy mobile apps reliably on new mobile hardware and software platforms.
Best practices for a future-proof mobile software testing strategy
How can mobile testing teams meet this challenge? Consider the following tips for ensuring that your mobile testing routine remains compatible with whatever the future will throw at it.
In a perfect world, you’d perform all of your mobile software tests on real devices. In the actual world, this is impractical in most cases, for a variety of reasons. Among them is the fact that if you attempt to test only on real devices, you may not be able to acquire new types of mobile hardware quickly enough to run mobile tests on it before your end users start acquiring the hardware. In order to avoid leaving your end-users with apps that haven’t been tested on their mobile devices, take advantage of simulators (or emulators — there is a difference between the two) to enable your team to run tests against newer hardware devices, even if you don’t have the hardware at hand.
If you want to test on the real versions of these devices when they are available, then by all means, do so. But don’t let an obsession with real-device mobile testing get in the way of supporting new hardware quickly.
Determine which mobile devices to test for
Just because a new mobile device hits the market doesn’t mean you need to test for it. Instead, you should decide which types of devices and operating systems to support based on those that your end-users are most likely to be using.
How do you figure that out, you ask? A good place to start is the Sauce Labs Mobile Application Testing Reference Guide, which contains several datasets regarding how the share of various mobile devices varies across different regions and demographic groups. If you know where your end users live and who they are, the guide will help you figure out which types of new devices you’ll need to incorporate into your mobile testing workflows, and which you can safely ignore.
Define which mobile features you support and which you don’t
Setting clear boundaries with developers ahead of time regarding which types of mobile features their apps will use can help to limit the scope of your mobile testing needs, and therefore make it easier to know which features in a new mobile device you need to test for and which you can ignore.
For example, you can determine which types of biometric features your apps can use, and which they will ignore. That way, if a new biometric feature rolls out that you’ve agreed not to use, the testing team won’t have to worry about writing tests for it.
This doesn’t mean that what you agree to support in a mobile app can never change. It just means that developers won’t decide to take advantage of a newly released mobile hardware or software feature without the testing team having time to prepare.
Setting these boundaries, by the way, requires that the software testing team be in constant conversation with developers. That’s where QAOps comes in. If you’re not familiar with QAOps principles, future-proofing your mobile testing strategy is one good reason to educate yourself about them.
Embrace continuous testing, automated testing and parallel testing
There are lots and lots of reasons to adopt continuous testing, which means testing at all stages of the delivery pipeline. The same is true of automated testing and parallel testing, which drastically reduce the time and effort it takes to run software tests.
Thus, you should already be practicing these three types of software testing virtues within your mobile testing strategy. But in case you need one more reason to embrace them, think about the importance of future-proofing your mobile testing strategy. By increasing your testing speed and agility, these practices make it easier to keep up with new demands on your mobile testing routine.
The mobile devices and environments that you have to test for today may not be the same ones you need to support tomorrow. An effective mobile testing strategy will allow you to adapt your testing routines and tools to meet changing needs.
Chris Tozzi has worked as a journalist and Linux systems administrator. He has particular interests in open source, agile infrastructure and networking. He is Senior Editor of content and a DevOps Analyst at Fixate IO. His latest book, For Fun and Profit: A History of the Free and Open Source Software Revolution, was published in 2017.