Having multiple testing frameworks at your disposal is essential for performing effective software testing and quality assurance. Why? Let me explain in this post…
Identifying Testing Frameworks
When I refer to multiple testing frameworks, I mean the ability to perform software tests using several open source test suites at once.
Sauce Labs is an example of a testing platform that supports multiple testing frameworks. On Sauce, you can use any or all of the following open source testing frameworks:
Selenium, which is probably the most widely used open source testing framework for browser testing. Selenium is the standby testing solution for many organizations—although it’s not usually the only one they use.
Appium. This testing framework is essentially an extension of Selenium to support mobile apps. Appium supports Selenium commands, but it is a distinct testing framework from Selenium because it is designed for a different set of use cases (namely, mobile app testing).
Espresso, an automated UI testing tool for Android. Espresso is Google’s offering for Android UI testing, which makes it an important testing framework to have in your arsenal.
Robotium, another widely used open source testing framework for Android apps.
The Benefits of Multiple Testing Frameworks
There is a fair amount of overlap between those test suites. For example, Espresso and Robotium both cater to the same class of applications—Android mobile apps. Appium also lets you test mobile apps, although it supports iOS apps in addition to Android.
So why would you want to have multiple testing frameworks? If one of these frameworks does everything you need, why worry about the others?
The Buffet Analogy
The answer is that your software testing experience should be like eating at a buffet restaurant.
The benefit of a buffet is that even if you go into the restaurant thinking you’d only like to have a certain thing for dinner, your choices are not limited if your desires change later. Maybe you arrive looking forward to Tex-Mex, then decide you’d like a little falafel to go with your tacos. Or perhaps you discover that the cheesecake you thought you’d have for dessert contains anisette, which you hate (seriously, does anyone actually like anisette?), so you need to change plans.
At a standard restaurant, it’s hard to pick and choose, or to change your decisions at the last minute. With a buffet, however, you have a great deal of flexibility.
From Buffets to Software Testing
I know. A buffet is a somewhat silly analogy for software testing. But it’s not totally off the mark. The point I am making is that just as a buffet gives you flexibility and agility in your dining choices, an automated testing platform that supports multiple testing frameworks allows you to pick and choose which frameworks to use, to deploy multiple frameworks at once, and to change your testing decisions on the fly.
Testing framework flexibility can be useful in a number of ways, including:
The ability to test web and mobile versions of an app at the same time. This is probably the most common reason for wanting multiple testing frameworks. It’s not efficient to have to use different platforms to test the web and mobile versions of the same app. It’s much better to be able to use Selenium and, for instance, Robotium alongside one another.
Getting the fastest test results. In some cases, you may find that certain tests for the same app run faster on one testing framework, but that that framework can’t be used in all cases. For example, Espresso is generally faster than Robotium, but Espresso doesn’t support as many releases of Android as Robotium does. If you can choose from among several frameworks and use them concurrently, you can optimize your results.
More thorough testing results. When you can use multiple testing frameworks, you don’t have to rely on the results of one testing tool alone. For example, Robotium may identify issues that Espresso misses, or vice versa.
Forward compatibility. You may only test a web app today, but in the future, you may need to test a mobile app as well. Having multiple frameworks available helps to ensure that you’ll have the testing resources you need to support future workloads without having to overhaul your entire testing process and environment.
The ability to use your team’s expertise to greatest effect. Some members of your team may be familiar with one testing framework, but not others. Assuming they can make a case for writing tests for the framework they know, they’ll be able to use that framework if you have multiple testing frameworks available. This is often better than forcing everyone to learn a certain framework because it’s the only one available to you.
Conclusion
To beat the analogy in this article dead, your software testing experience should be like the numerous options in a buffet. You shouldn’t have to commit yourself to a certain testing framework, or have to use only one framework at a time. When you have multiple testing frameworks available, you can optimize your testing workflows to get the fastest, most thorough results.
Chris Tozzi has worked as a journalist and Linux systems administrator. He has particular interests in open source, agile infrastructure and networking. He is Senior Editor of content and a DevOps Analyst at Fixate IO.
