Geographic Considerations for Mobile Device Testing
In an ideal world, you’d test all versions of all operating systems and devices for all of your mobile apps.
In the real world, however, testing everything just isn’t feasible (in most cases). That’s why you need to plan a mobile testing strategy that helps you cover the widest range of use cases for your target user base in the most efficient way possible.
Factoring geography into your test strategy is an important part of striking this equilibrium. By considering how the needs of your users vary based on geographic location, you can develop the most effective testing strategy for mobile apps.
Device Operating Systems
One of the ways in which geography plays into mobile device testing is that different mobile operating systems are favored in different parts of the world. According to some statistics, for instance, about half of the mobile devices that are used in the United States run the iOS operating system. In India, however, that number drops to around 3%. Therefore, if you were building a mobile application that was designed primarily for use in India, you would probably want to focus most if not all of your development and testing efforts on iOS competitors.
Another aspect of the testing process that must be considered is that of localizations. Localizations refer to region-specific settings within the device’s operating system. Suppose, for instance, a particular device is configured for use in the United States. The device might be configured to use the English language, and to use the American dollar for any currency-based operations. Likewise, the device would probably also be set up to use the English system of measurement (as opposed to the metric system), and to display the date in month, day, year format.
Localizations are usually configured by the end user as a part of the device’s initial setup process. Even so, application developers need to ensure that the applications that they create will work properly with the localization settings for the regions within which the application is expected to be used. Just because an application works perfectly in the United States, for example, does not guarantee that it will work properly in Europe. If the application is specifically designed to display financial data in American dollars, for instance, it might not display Euros properly. In any case, it is important to build a mobile device testing strategy using the device localization settings that correspond to the region where the application is expected to be used.
Operating System Version
Another key consideration is the operating system versions that are popular in the region where the application is expected to be used. The latest device operating system might not be as easy to acquire in a developing country as it is at home. Therefore, apps should be tested against the operating systems that are popular in the region where the application will be used.
Fortunately, there are resources available to help you figure out which mobile operating systems you should be testing against. One of the sources of information is Device Atlas. Device Atlas provides mobile device usage statistics for countries around the world. In 2017, for example, Android 5-5.1 (Lollipop) devices are slightly more popular than Android 6 (Marshmallow) devices in India. In Germany, however, Android 6 (Marshmallow) devices are far more popular than other Android versions. For more on this, see: https://deviceatlas.com/blog/android-versions-market-share-2017.
One more consideration that must be taken into account is that the available bandwidth for mobile devices can differ considerably in different parts of the world. Furthermore, you cannot necessarily assume that 4G speeds abroad are the same as they are at home. In India, for example, 4G mobile broadband is widely available. Because of bandwidth saturation, however, 4G connectivity is reported to be only marginally faster than 3G. According to the Economic Times, the average global 4G download speed is 16.2 Mbps, but in India, the average 4G download speed is a mere 5.1 Mbps.
I am not trying to pick on India, but rather want to make the point that just because an area has widespread 4G coverage, it does not mean that mobile network performance will be similar to what you are used to at home. Depending on the region, performance can be much lower (or higher) than what you are used to.
When performing mobile device testing, it is important to take into account any differences in connectivity speeds. An application that seems fluid and responsive at home may end up being painfully slow when used abroad, due to differences in the available bandwidth.
When building mobile apps that are designed for international use, it is important to remember that the app might perform differently in other parts of the world. As such, it is crucial to do a lot of real mobile device testing in a way that emulates the conditions that are expected in the app’s target market. This might include the device OS version, localization settings, and even the available bandwidth.
Need help getting started with a mobile app testing strategy? Check out the Sauce Labs Mobile Testing Reference Guide, which offers comprehensive guidance based on our analysis of mobile usage trends.
Brien Posey is a Fixate IO contributor, and a 16-time Microsoft MVP with over two decades of IT experience. Prior to going freelance, Brien was CIO for a national chain of hospitals and healthcare facilities. He also served as lead network engineer for the United States Department of Defense at Fort Knox. Brien has also worked as a network administrator for some of the largest insurance companies in America. In addition to his continued work in IT, Brien has spent the last three years training as a Commercial Scientist-Astronaut Candidate for a mission to study polar mesospheric clouds from space. You can follow Posey’s spaceflight training at www.brienposey.com/space.
- Accessibility Testing
- Appium Resources
- Best Practices
- Continuous Delivery
- Continuous Integration
- Continuous Testing
- Cross Browser Testing
- Guest Blog Posts
- Load Testing
- Machine Learning
- Mobile Development & Testing
- News & Product Updates
- Open Sauce
- Open Source
- Performance Testing
- Product Updates
- Quality Assurance
- Quality Engineering
- Sauce Product Info
- Security Testing
- Selenium Resources
- Software Development & Testing
- The Story of Sauce