Mobile Testing Basics: Manual vs. Automated Testing

Deciding to build a mobile app is an important step forward to meet customer’s expectations and quality plays an important role in today’s mobile app world. One way to retain users is to provide better quality than other similar apps, that can be achieved through testing. In this article we’re going to walk you through mobile testing basics!

Mobile testing basics

When it comes to (mobile) testing there are two main approaches: manual testing and automated testing. Both of these approaches are valid and have the goal to test mobile applications comprehensively in different ways, and in different moments of an app’s lifecycle – ultimately increasing their quality.
Let’s have a closer look at these two testing practices.

Manual testing

What is manual testing?

Manual testing is the most common testing approach for both mobile and desktop applications. Testing manually is what gives you the real feeling of how it is to actually use the application. You can see if the buttons are in the right position, if they are big enough, if the colors look good together etc. Moreover you can test actions e.g. launch, login, search. These things can also be tested with automation.

Manual testing can be done with either a real device in your hand or with an emulator/simulator, but real devices will give results which are more similar to what your users will experience.

Why do I need manual testing?

Manual testing ensures the functionality of an application and tests its features. The tester plays the role of a user trying out everything in that application, doing typical actions a user would, to see if or when the app crashes. With a manual testing you can also test things like performance, battery drainage or overheating and fix them before release.

How many devices to I need for testing?

The ideal response would be “on every device”, but that is not actually doable. There are more than 24000 different Android devices, so you should test on the most sold/used devices either in your local market or worldwide. A real number of devices, based on our research, would be 10-20 devices, depending on how big your target market is.

Automated testing

What is automated testing?

Automated testing is the process in which pre-scripted tests are executed on an app before it is released into production. Automated testing helps you find flaws in your app quicker. Test automation can be used to run tests that are repetitive, that need to be run periodically and that can help you find bugs in early stages. While the aim of manual testing is to test the so-called “user experience”, automated testing aims to test all the functionalities that characterizes an app.

Why do I need automated testing?

You need automated testing to speed up your testing process as well as the whole development process. Test automation allows you to:

  • Test all those functionalities that are repetitive and therefore error prone if performed manually; test cases that have a predictable outcome;
  • Complicated and tedious test cases and
  • Most important: you can test on a higher number of mobile devices simultaneously, saving time.

How many test cases do I need for a basic set of tests?

Depending on the complexity of your app the number of “basic” test cases may vary. From our experience, the usual test case volume is about 20-30 test cases for each OS. So if you have an Android app and one iOS app, you will need around 40-60 test cases. You should also test on a wider range of mobile device, say around 50 devices.

Which approach to choose

Introducing integrated testing

We, at TestObject, call integrated testing the approach that combines manual and automated testing for maximum efficiency, time and money saving. Neither approach should be skipped, otherwise the quality of your mobile app might not be what you expect. The ideal testing ratio is 80% automated testing and 20% manual testing. Based on our research, combining the two approaches in that ratio can help you save up to 70% of your money and time spent testing. This percentages can change based on the complexity and the concept of your app.

integrated testing

Testing comprehensively through the app’s lifecycle

The picture below shows what tests should be performed through manual testing and which one through test automation. An agile approach is also advised when developing mobile applications.

Tests to perform in the different life-cycle stages of an app

Written by

Ely Hechtel

Topics

Manual/Live testingAutomated testing

Categories