Mobile Testing Basics: Manual vs. Automated Testing
Deciding to build a mobile app is an important step forward to meet customer’s expectations and quality plays an important role in today’s mobile app world. One way to retain users is to provide better quality than other similar apps, that can be achieved through testing. In this article we’re going to walk you through mobile testing basics!
Mobile testing basics
When it comes to (mobile) testing there are two main approaches: manual testing and automated testing. Both of these approaches are valid and have the goal to test mobile applications comprehensively in different ways, and in different moments of an app’s lifecycle – ultimately increasing their quality.
Let’s have a closer look at these two testing practices.
What is manual testing?
Manual testing is the most common testing approach for both mobile and desktop applications. Testing manually is what gives you the real feeling of how it is to actually use the application. You can see if the buttons are in the right position, if they are big enough, if the colors look good together etc. Moreover you can test actions e.g. launch, login, search. These things can also be tested with automation.
Manual testing can be done with either a real device in your hand or with an emulator/simulator, but real devices will give results which are more similar to what your users will experience.
Why do I need manual testing?
Manual testing ensures the functionality of an application and tests its features. The tester plays the role of a user trying out everything in that application, doing typical actions a user would, to see if or when the app crashes. With a manual testing you can also test things like performance, battery drainage or overheating and fix them before release.
How many devices to I need for testing?
The ideal response would be “on every device”, but that is not actually doable. There are more than 24000 different Android devices, so you should test on the most sold/used devices either in your local market or worldwide. A real number of devices, based on our research, would be 10-20 devices, depending on how big your target market is.
What is automated testing?
Automated testing is the process in which pre-scripted tests are executed on an app before it is released into production. Automated testing helps you find flaws in your app quicker. Test automation can be used to run tests that are repetitive, that need to be run periodically and that can help you find bugs in early stages. While the aim of manual testing is to test the so-called “user experience”, automated testing aims to test all the functionalities that characterizes an app.
Why do I need automated testing?
You need automated testing to speed up your testing process as well as the whole development process. Test automation allows you to:
- Test all those functionalities that are repetitive and therefore error prone if performed manually; test cases that have a predictable outcome;
- Complicated and tedious test cases and
- Most important: you can test on a higher number of mobile devices simultaneously, saving time.
How many test cases do I need for a basic set of tests?
Depending on the complexity of your app the number of “basic” test cases may vary. From our experience, the usual test case volume is about 20-30 test cases for each OS. So if you have an Android app and one iOS app, you will need around 40-60 test cases. You should also test on a wider range of mobile device, say around 50 devices.
Which approach to choose
Introducing integrated testing
We, at TestObject, call integrated testing the approach that combines manual and automated testing for maximum efficiency, time and money saving. Neither approach should be skipped, otherwise the quality of your mobile app might not be what you expect. The ideal testing ratio is 80% automated testing and 20% manual testing. Based on our research, combining the two approaches in that ratio can help you save up to 70% of your money and time spent testing. This percentages can change based on the complexity and the concept of your app.
Testing comprehensively through the app’s lifecycle
The picture below shows what tests should be performed through manual testing and which one through test automation. An agile approach is also advised when developing mobile applications.
- Accessibility Testing
- Appium Resources
- Best Practices
- Continuous Delivery
- Continuous Integration
- Continuous Testing
- Cross Browser Testing
- Guest Blog Posts
- Load Testing
- Machine Learning
- Mobile Development & Testing
- News & Product Updates
- Open Sauce
- Open Source
- Performance Testing
- Product Updates
- Quality Assurance
- Quality Engineering
- Sauce Product Info
- Security Testing
- Selenium Resources
- Software Development & Testing
- The Story of Sauce