Easily Run 50,000 to 200,000 Tests with Parallel Test Execution in Automation

Posted Mar 12th, 2019

At some point in test automation, every organization runs into a common problem: too many automated UI tests take too long to execute. But there is a solution…executing tests in parallel.

In fact, this is the most powerful way to speed up suite execution. In a recent test, I was able to reduce my test suite execution time by over 500%. How did I do it? Simply by combining the Sauce Labs Continuous Testing Cloud with a few industry best practices.

Sauce Labs is optimized for testing in parallel, and at any scale, across many different browser, OS and device combinations. I was able to dramatically reduce my time to test by using the following best practices for parallel testing. Here's what I did. 

Utilized Atomic Testing. I only tested a single feature in the application at a time, which allowed me to control the state of the web application. When combined, testing individual features within the application will equate to end-to-end testing. This is a ‘must do’ for ultimate time savings in parallel test execution.

Avoided the static keyword. Nothing ends dreams of parallelization faster than an improperly placed static keyword. A static browser object makes it impossible to run multiple tests at the same time as each test will try to use the same exact object. One test might be trying to open a browser while another might be trying to locate an element. As a result, the different actions interfere with each other because they are acting on the same exact object. So, I made sure not to do this.

Made the tests autonomous. An autonomous automated test is one that doesn’t rely on the outcome of another test. By not forcing tests to run based on the outcome of another test I could execute tests in parallel and without fear of false fail.

Managed the test data. The best way to manage test data for parallelization is to use just-in-time test data. This is the creation of test data before the test starts, the use of that test data in the test case, and then the clean up the test data right after. Let’s say you need a test user with specific attributes in order to navigate through an application. If such a user isn’t created dynamically, then one test might leave the user in an undesirable state. The next test then tries to execute, expecting the user to be in a default state, and fails because it is not. Using just-in-time test data I resolved this issue.

Parallelization is critical to running suites quickly. With Sauce Labs and the steps above, you’ll do just that. For more information and specific instructions, check out my Tech Talk on this topic or review our data sheet.  

Written by

Nikolay Advolodkin


Automated testingContinuous testing