If you’re like most app development and QA teams, you’ll discover that adopting automated testing requires clearing some hurdles. This is because, like most innovations, test automation creates some challenges that organizations need to overcome before they can reap the full benefits of an automated testing strategy.
The most common pain point that QA teams tend to experience when they move to an automation-based testing strategy is a lack of expertise with test automation.
This challenge is to be expected. If your QA engineers were previously accustomed to performing testing by hand, they likely lack experience in writing automated test scripts or using automated testing frameworks, like Selenium and Appium.
The obvious solution to this challenge is to educate your QA engineers, of course. But you don’t have to limit your approach to forcing your QA team to take tutorials on writing test scripts or pore over Selenium documentation.
You might also think about how to leverage the coding skills that your organization already possesses to help your QA team learn to write automated tests faster and more effectively. Ask your developers to assist in the automated testing learning process. The coders you have on staff may not have backgrounds in QA engineering, but they should be able to wrap their heads easily around writing test scripts and learning a new scripting framework. By involving developers in the automated-testing migration process, you can speed your QA engineers’ ability to gain the skills they need in order to take full advantage of automated testing.
Another common challenge that organizations face when moving to automated testing is a lack of infrastructure for executing their tests.
To solve this problem, you have two options. The first (and the one to which you might be most tempted to turn) is to build your own automated testing environments by setting up software like Selenium server on your own on-premise infrastructure.
That approach works, but it’s less than ideal, for two reasons: First, setting up automated testing software will take time, especially if your team doesn’t have prior experience configuring these tools. Second, an on-premises test infrastructure is difficult to scale. If you plan to increase the number of automated tests that you run, you’ll have to add more infrastructure, which will be costly and take time.
The other, better approach is to use a cloud-based testing solution, where the environment is already set up to let you run tests using any framework you choose. An automated testing cloud is also highly scalable; you can use as much infrastructure as you need to run as many tests as you want at the same time.
The market offers a plethora of test automation frameworks to choose from, and the decision isn’t always easy. As you evaluate mobile testing frameworks to determine what's suitable for your organization, you should ask some key questions to help uncover your needs and understand the impact of a new tool on your organizational processes, tech stack, and teams.
The following are popular test automation frameworks:
Appium - Appium is a leading open-source test framework that allows for cross-platform (mobile) native test automation, as it supports both iOS and Android platforms. It was derived from Selenium to extend automated testing functionality to mobile apps.
XCUITest - XCUITest is the testing framework that ships with Apple’s XCode development system, and as such, it is the most popular framework for performing basic unit-level testing of iOS apps.
Espresso - Espresso is the testing framework that comes built into Android Studio and is designed specifically for functional testing of Android applications.
Puppeteer - Puppeteer is a great alternative to the newer and less stable Playwright. Puppeteer is the go-to test automation framework for developers who only need to support Chrome and Chromium. For cross-browser tests, frameworks like Playwright provide a similar experience.
EarlGrey - EarlGrey is an open-source iOS UI automation framework developed by Google. It supports only iOS and hence requires tests to be written in Objective-C or Swift. Some benefits of EarlGrey include easy integration into iOS, simplicity in writing tests, and enhanced built-in synchronization features which help to increase test stability.
Not all tests should be automated, but most unit, integration, and performance testing can be easily automated. Consider the following criteria when deciding which tests to automate:
High value and business critical: Tests that ensure your app keeps running smoothly and uninterrupted, such as smoke tests, sanity tests, and regression tests.
High volume, repeatable, and/or tedious: Tests scripts that are time consuming or need to be repeated many times, such as testing different browser/OS combinations and running high-volume batch tests.
Predictable results: Tests that are both repeatable and have predictable outcomes, such as stress tests and load tests.
Frequently used functionality: Tests frequently used functionality that comes with high-risk conditions for both the device and sensitive user data, such as loss of functionality (e.g., loss of data connection), unexpected high data usage, battery draining 2-3 times faster than normal, and security or privacy exposure.
Impossible to perform manually: Tests that can’t be performed manually. For example, a manual test may return an issue that you’re unable to diagnose. Automated tests can not only identify an issue but also indicate the cause of the issue.
Test data and environments: Tests that automate tasks like setting up environments and test data.
Your mileage will vary, of course, but in general, these pointers are a good place to start when you’re deciding which tests are the best candidates for automation.
Chris Riley (@HoardingInfo) is a technologist who has spent over 15 years helping organizations transition from traditional development practices to a modern set of culture, processes and tooling. In addition to being an industry analyst, he is a regular author, speaker, and evangelist in the areas of DevOps, BigData, and IT. Chris believes the biggest challenges faced in the tech market are not tools, but rather people and planning.