Learn how to identify which types of environments to prioritize as part of a software testing strategy.
Posted Jul 24, 2018
Learn how to identify which types of environments to prioritize as part of a software testing strategy.
In a world where the Android ecosystem alone has spawned 24,000 different types of mobile devices—to say nothing of iOS, a smaller but still diverse device market—deciding which devices to support when writing and testing software can be a challenge. When you factor in multiple types and versions of operating systems and web browsers, building a software testing strategy that covers as many software environments as necessary may feel like a decidedly impossible battle.
Faced with these challenges, some organizations are tempted not to test their software for a broad range of environments at all. Others may lack a systematic software testing strategy, relying on ad hoc tests and suffering shortcomings in software quality as a result.
Yet while the dizzying number of different combinations of device types, operating systems and browsers may seem impossible to manage, the fact is that modern software testing solutions make it feasible for software delivery teams to tame this challenge. While testing software for every possible environment is not realistic for most organizations, it is eminently possible to develop a software testing strategy that offers broad coverage but is also practical to implement and manage.
This guide explains how to identify which types of environments to prioritize as part of a software testing strategy, as well as how to implement a testing routine that provides testing coverage that is as broad as your business needs.
When we mention software environments, we mean the type of environment in which an application might run. A variety of factors determine what a given end-user’s software environment looks like and how it causes an application to behave.
The three main factors in shaping a software environment are:
It is worth noting that other variables can impact application behavior, too, such as the version of the various software libraries that an application depends on and language and region settings on the host device. Your business may or may not choose to incorporate these variables into its testing strategy, depending on whether they are important for your target market or not. If you only release software for English-speaking users in North America, for example, testing for different language and region settings is not likely to be a priority.
Given that hundreds of thousands of possible device/operating system/browser combinations exist, it is not realistic to attempt to test your applications for every possible environment. Instead, your software testing team should identify the types of environments that are most important for meeting your business needs, based on the following factors.
The market share of various devices, operating systems and browsers varies widely between different regions. iPhones are more popular in North America than in the world as a whole, for example.
Identifying which types of environments are most common among your target users based on the regions in which they live is therefore a useful first step in deciding which environments to prioritize for testing. The “Mobile Reference Testing Guide,” which offers data about environment popularity across different regions, is a useful resource for this task.
Another obvious factor to consider is the types of applications you plan to build. If you only build native mobile apps that don’t depend on a web browser, for example, you eliminate the need to test for different browser versions—although you may also limit the portability of your application, because applications that depend on functionality that is native to a specific device are harder to migrate to other devices than those that run in a web browser.
Striking the right balance between application functionality and ease of testing is thus another useful step in planning a software testing strategy.
Do you build software for all types of devices? Or do you target, for example, only mobile devices and PCs? How might your device support needs change in the future? Obviously, there is no need to test for devices that you don’t currently support.
That said, it is wise to future-proof your testing strategy by ensuring that you can easily extend testing support to more devices if needed in the future. Testing for as many types of devices as possible can also provide business benefits by enabling an omnichannel presence, as explained in the following section.
While it will be necessary to limit the number of environments that you support based on the factors described above, this does not mean that you should not strive to test for as large a combination of devices, operating systems and browsers as possible.
The benefits of broad test coverage for your business are many:
It may not seem feasible to test applications for a broad range of environments, especially if your business has limited IT resources. By taking the right approach, however, it is quite possible to achieve broad environment coverage without overextending your testing team.
A test strategy that enables broad coverage in an efficient way is defined by three main factors:
A certain amount of picking and choosing will always be necessary when deciding which types of software environments to include in your testing strategy.
However, by designing a testing process that makes use of cloud-based test infrastructure and automated testing, it is possible to achieve broader test coverage—and, by extension, to maximize your business’s ability to support new and existing users no matter which types of devices, operating systems and web browsers they might use.
You can, in other words, have the best of both worlds: cost-efficient software testing as well as broad environment coverage that not only supports your business’s needs today, but can also scale as your business grows.