Mobile testing is the process of testing applications that are designed for mobile devices, with a focus on elements such as functionality, usability, performance, and security. The need for a comprehensive mobile testing strategy is growing in importance due to our increasing reliance on handheld devices. In November, 2016 the number of Internet users on handheld devices far exceeded that of traditional desktop users for the first time ever, and this trend will only continue to grow. It has become an accepted standard that personal and professional interactions are now conducted on these portable devices while we are on the go. Whether it be social networking, gaming, banking, e-commerce, or ride-sharing, we are always connected and do everything on our phones.
This is why a well thought-out mobile testing strategy is vital for your business and for your users. In order to beat competitors, you have to nail the first impression of the user experience because studies show that the majority of users will simply delete and not reinstall an application if they encounter any sort of performance, usability, or security issue. It is more costly to a business to retain a dissatisfied customer than to obtain a new one.
That being said, testing mobile applications comes with a lot of challenges unforeseen in traditional application testing, namely due to the thousands of combinations of mobile devices, the operating system an app runs on and OS versions. In addition, you have to adjust your mobile testing strategy for native, web, and hybrid applications. Unlike traditional web applications, mobile applications are deployed in a less flexible and less frequent manner, and have to adhere to app store policies, which leaves even less room for failure.
The best way to tackle these challenges is to develop a clear and extensive mobile testing strategy that incorporates business objectives and customer expectations before deployment occurs. Mobile testing is still fairly new, and best practices are still being assembled, but this article will aim to explore some important elements of a mobile testing strategy that is effective in the current landscape.
Mobile Testing Tip 1: Test against a broad range of devices
Your testing strategy needs to adapt to the fragmented mobile market, which means that each application needs its compatibility to be tested against multiple devices, platforms, and operating systems and versions. This is a particularly acute problem for Android devices. Test against all stable versions of operating systems and use an abundant mix of screen sizes and resolutions. Understand what your users are using, what the most popular combinations currently are, and what effect that can have on the application. Then, establish relevant tests across all target devices.
Mobile Testing Tip 2: Find the right balance between automated and manual testing
Automated and manual testing are both required for an effective mobile testing strategy. Although the idea of automating everything is enticing, some level of manual testing is always necessary, and you need to find the right balance for your needs. Automated mobile testing can be applied to cumbersome tasks that function in a fairly standard way and is very useful for regression testing at scale, as well as for load tests.
Mobile Testing Tip 3: Remember to test on a mix of emulators/simulators and real devices
Testing on simulated devices tends to be fast, and is probably the basis for your testing strategy, particularly if you’re running automated tests as part of a CI/CD pipeline. In contrast, testing on real mobile devices doesn't scale very well, but it is absolutely vital to have the ability to do real-device testing when necessary as part of a comprehensive mobile testing strategy because it provides real information on the user experience. Some conditions are not able to be reproduced accurately on a simulator, such as power consumption and memory restrictions. User experience components that can only be observed on an actual device include touch response, swiping instead of clicking, optimal button sizes, and the color/brightness of the application under different lighting conditions. In order to test effectively for these types of features, a mix of simulators and actual devices is crucial (read more).
Mobile Testing Tip 4: Experiment under varying network connectivity
Mobile users are not likely to have access to network connectivity at all times, and this real-world condition needs to be simulated during your testing. Observe how the mobile application behaves under different network situations such as congestion or high latency and how it handles different connectivity modes (WiFi, LTE, 4G, airplane mode, etc.) and the impact on usability.
Some useful approaches include identifying the application's target audience and the type of connectivity they would be likely to have - and testing mostly on WiFi, but using a network simulator to test against different speeds, bandwidths, etc.
Mobile Testing Tip 5: Assess all security angles
A lot of data is stored on our handheld devices, and our devices are likely to be almost always online and traceable. Your mobile testing strategy might immediately fixate on functionality and usability, but security is of equal testing importance. Test for things like how well the application integrates with other applications, whether communications are secured and encrypted, whether malware can be installed, and whether authentication procedures are secure. In terms of data storage, look into what kinds of data the application saves, collects, uses, sends, and has access to. Devise tests that will attack the application to detect vulnerabilities.
Mobile Testing Tip 6: Consider cloud solutions
Adopting modern testing frameworks and platforms can greatly improve your mobile testing strategy and provide many benefits, such as scaling your testing, increasing the variety of test scenarios, supporting complex applications, and providing real-time results. Cloud testing solutions can be an effective and efficient way to combat the overwhelmingly fragmented nature of the mobile landscape. Instead of building your own on-premises testing infrastructure, you can leverage cloud-based mobile testing platforms like the one offered by Sauce Labs, which provides the tools necessary to reduce your product's time to market and improve overall software quality.
Mobile testing poses a lot of unique challenges, but a well-formulated mobile testing strategy with the aforementioned considerations in mind will guide you on the path to success.
Want to learn more? Check out this on-demand webinar, "Simplify Your Mobile Testing Strategy".
Daisy Tsang is a software developer currently based in Berlin. She has worked at companies in Canada and Germany in the past few years, loves learning about new technologies, and is particularly interested in the open source space. Her non-technical interests include knitting, baking, and learning new natural languages.