If you develop mobile apps you will need to hire a QA manager at some point. They will help you manage the test cases, improve and structure your test process and ultimately grow your business. Here are 10 questions you should ask candidates during your job interview.
Let’s talk about what a mobile QA manager does first
So let’s start with this before moving on to the questions a qualified QA manager should be able to answer. Ideally a QA manager is the one person that ensures that a product meets the required quality standards, mainly in terms of reliability, usability and performance. This person should also be able to create a straightforward testing strategy as well as organise the test cases.
Now we can start with the questions that should be asked during an interview. You can add the questions below to your list. The interview should not consist only of these question and they can be asked in the order you prefer or that fit your interview schedule best.
1. In a complex mobile testing environment, what do you think are the the biggest challenges?
Ideally your candidate should mention four “typical” mobile testing challenges like:
Android device fragmentation, mainly regarding OS and OS versions, caused by manufacturer’s specific differences based on hardware, price and local market
Screen size fragmentation
Importance of localization
Importance of different mobile networks and user mobility
2. As a QA Manager, what you think is the best approach to test apps extensively?
Here you would expect the candidate tell you about his extensive knowledge about manual testing and automated testing and how it would be a good idea to combine both of them to save time and money.
3. Still staying in the topic of the previous question (manual and automated testing), what test cases should be automated and which ones performed manually, in your opinion?
At this point there are certain guidelines to decide which test cases should be automated and which shouldn’t.
The candidate should talk about a few of these points when talking about test automation:
Automate the most frequent test-cases: all the use cases that are frequently performed manually – could be automated instead
Automate test cases that are easy to automate
Automate test cases that have predictable and easily verifiable results
Automate the most tedious manual test cases
Automate test cases that are impossible to perform manually
Automate test cases that run on several different hardware or software platforms and configurations
When talking about manual testing these are important points:
Manual testing should be performed whenever the test case is too complicated and would take a lot of time to establish an automated test case for.
If certain parts of the app will be changed in the near future, it is advisable not to start writing scripts for these test cases, because they will be unusable very soon.
4. What mobile test automation frameworks are you familiar with and could you tell us about their pros and cons?
This question has a direct correlation to automated testing, so questions with a flowing topic should be asked in sequence. If the candidate nominates a testing framework he should be to talk about pros and cons. It would be beneficial if they mentioned well known frameworks like UI Automation, Robotium, Calabash, Appium, and Espresso, even if they will not be the ones who write the scripts.
5. How can mobile test automation be leveraged effectively?
There is no textbook answer here, because it depends on the implemented strategy, but a good answer should highlight some of these aspects:
Test automation can deliver information fast, but it does not have a great value if the test results do not reach a developer
A QA manager’s job would be to provide the right developer with the right information (as many as possible e.g. screenshots, logs and videos) so they can fix the issues
CI integration would be a valuable tool: it would help you integrate the test infrastructure into your own development pipeline to help information flow faster
6. How would you define a good test case? And a bad test case? Can you give us examples?
Your future QA Manager should be able to give you a clear example of a bad and a good test case. A good test case is defined by a short but clear description of what is expected from the app.
The example for both could look like this:
Bad test case
There can be two main bad test case scenarios. The first one would be very undescriptive, e.g. install, launch and login. In this case you don’t really know what to expect exactly from the test. The other bad test case would be a test case that tries to test many different functionalities at the same time. The result would be that you don’t concentrate on just one aspect of the app at a time and you might be missing other important functionality.
Good test case
Here is an example of a good test case: install and launch the application. Try to login with user info@interview.com with password “abcd”, wait 5 seconds for the app to respond and shows you the app interface after the login.
7. How do you decide on mobile devices, OS and OS versions for testing?
The device choice is always tough, but there are some ways to decide which mobile devices are best suited for your app testing. The answer of your QA manager should include the fact that most apps nowadays are global apps, that means that you should ideally use global sources if your app is meant to be available worldwide, otherwise the local market should be researched. That can be done by analyzing the most used/sold devices either globally or in the local market.
One other criteria the candidate could point to is to analyze the app download statistics, which contain information about the most popular devices. They should also think of checking the app’s reviews to find out if users have issues happening on specific devices. If the app hasn’t been released yet, you obviously don’t have any data to carry out this strategy.
Ideally the candidate will tell you to combine both popular devices and app stores statistics based on the functionality of the app and the target audience.Device trends could also be a good criteria as well as looking at which devices are being announced and how much expectation there is on those device from users.
8. What do you think about Continuous Integration in the development process?
The candidates answer should include that continuous integration is an important part of mobile application testing, because the development process should be an agile one. Ideally they have already had experience with Jenkins or Travis CI, or other, and they can give you their opinion.
9. Imagine you are the first QA manager joining our company. What would be the first three things you’d do?
Many companies, especially smaller ones, do not have a formal testing process, so it is important for the QA Manager to ask questions regarding how the testing process works now. They should then start to outline the things they would do to make the process straightforward, and how they would start introducing a proper testing plan and strategy.
Small companies can start with using a spreadsheet to write about 20 test cases (depending on how complicate the app is and how comprehensively it needs to be tested) and start from there. The test cases will be building up over time leading to inevitable choice of test management tools, tracking tools, CI etc.
10. If you found yourself working on a project for which no test automation has been done yet, and you only had a very limited amount of resources, what kind of tests would you aim to set up?
In this case the candidate would think about the most important things to be tested and would suggest to test critical features or known issues the app already has (if it’s been released). They would also advise to do smoke testing, which will reveal simple failures severe enough to know that the app or the new feature is not ready for release yet.
