There are lots of reasons for running software tests in the cloud. Some, such as faster test results due to the ability to test in parallel, are obvious, and align with the benefits of cloud computing in general.
Other advantages of cloud-based testing, however, are less obvious, especially if you are not already running tests in the cloud.
To gain a full understanding of why you would want to perform cloud testing, let’s take a look at all of the benefits of cloud-based software tests.
Speed and Cloud Testing
Speed is perhaps the most obvious reason to perform cloud tests. Cloud-based software tests tend to run faster than those that are hosted on local hardware for a couple of key reasons.
The first and most significant one, as mentioned above, is that a cloud-based test grid enables you to run as many tests in parallel as you want. When you don’t have to wait for one test to complete before you have infrastructure available to run the next one, you get results faster.
Cloud testing also increases speed because the capacity of the test infrastructure is virtually unlimited. If you have a particularly resource-hungry test to run, you won’t have to sacrifice speed to complete it if it runs in the cloud.
Breadth of Cloud Testing Coverage
Another relatively obvious benefit of cloud testing is that a cloud-based test grid makes it easy to test your application against a very large variety of environments without having to set up any infrastructure yourself.
If you rely on on-premises test infrastructure, your tests will probably be limited in scope, because you won’t have the time or resources to set up as many devices and software environments as those available to you in the cloud.
Extended Debugging and Cloud Testing
In addition to giving you a turnkey infrastructure for hosting Selenium tests, advanced cloud-based test grids also help to streamline the process of troubleshooting tests. That is because a test cloud can collect and display data about test results that can help QA teams to understand why some tests are “flaky” or intermittently fail for reasons that are hard to track down. This is the type of insight that Sauce Labs offers through its Extended Debugging feature.
You could attempt to collect and interpret this data from a local test grid, of course. But doing so would require a great deal of effort. And because your local test grid is probably smaller than a test cloud, it might not generate enough data to make analytics-based debugging work.
Video of Cloud Tests
Advanced test clouds also enable the ability to record videos of tests in action and play them back at a later time. This type of functionality can be tremendously useful when you need to troubleshoot a test or want to gain deeper insight than test results alone can provide.
True, you could use screencasting tools to record tests on local infrastructure. But automating recordings in this way would take some know-how, and this solution would not be practical if you have large numbers of tests to record.
Your Test Grid Evolves Automatically
When a new version of a testing framework debuts, or you need to add tests for a new type of device or operating system, configuring a local test grid to support the change will require a significant investment of time. You may also need to acquire new hardware.
With a cloud test grid, updates happen automatically. New devices are added as they appear on the market, and cloud test vendors handle software updates for you. This means you can spend more time improving your apps, while someone else keeps your test grid up-to-date.
Real Device Testing
Your ability to perform tests on real devices as opposed to using simulators on a local grid will almost always be limited. Most organizations can’t afford to purchase and manage hundreds of different devices for hosting tests.
In contrast, cloud testing gives you the option of performing tests on real devices if you wish, while also providing access to simulators.
Conclusion
While local software testing may make sense if your testing needs are very small or temporary, cloud testing offers a number of clear advantages for organizations that have serious tests to run on a recurring basis as part of continuous delivery pipelines. While you could get the job done with a local test grid, tests will take longer to complete, they will be harder to interpret and troubleshoot, and your testing coverage will not be as large as with cloud tests.
Chris Tozzi has worked as a journalist and Linux systems administrator. He has particular interests in open source, agile infrastructure and networking. He is Senior Editor of content and a DevOps Analyst at Fixate IO. His latest book, For Fun and Profit: A History of the Free and Open Source Software Revolution, was published in 2017.
