How to Optimize a Test and Make It 560% Faster

Posted Oct 8, 2020

Sauce Bot Training Under Construction

Deep in the implementation of every automated UI test lives the potential to turn something simple into something slow and unreliable—simply by adding extra Selenium commands. The data clearly show that longer tests are less likely to pass. In this case study, I will show you how to optimize a test and make it 560% faster. We will do this by tackling inefficient use of Selenium commands.

Before optimizations

Too many Selenium commands in a test can lead to tests that are dramatically slower and less reliable. Every Selenium command carries the potential for an invalid element locator or a synchronization issue. 

Below we have a table that shows an automated test that was designed to click two radio buttons. V1 test is making 157 Selenium commands and takes 95 seconds to execute. After optimizations, the same exact test executes 9 Selenium commands and runs in 17 seconds (5X faster).

Nikolay_Selenium_1

Nikolay Selenium 2

Total test run time V1

Nikolay Selenium 3

Number of web requests before optimizations (V1)

What was the purpose of this test?

The purpose of this test was to simply click two radio buttons on this URL

Nikolay Selenium 4

Identifying the inefficiencies

The major contributor to the slow test time of such a simple test was the large number of unnecessary web requests.

Attempting to click first radio button

After loading the URL, the test takes ~60 seconds searching for random elements without taking a single action.

Nikolay Selenium 5

Unnecessary commands start at the 15-second mark and continue until 1:19, at which point the element is finally located and clicked.

Nikolay selenium 6

Potential Waste: 60 seconds on 1 element

Attempting to click second radio button

At 1:23, the test starts looking for the Female radio button.

Nikolay Selenium 7

And finally clicks it at 1:28:

Nikolay Selenium 8

...but not before making 16 more Selenium commands to check whether the element is displayed or selected.

Nikolay Selenium 9

Potential waste: 3 seconds for every element.click()

Post-click() Selenium commands

Finally, there is an interesting pattern of wasted web requests that happens after every element click(). We can see that below:

Nikolay Selenium 10

It’s understandable that these are being performed to validate that the element is in the correct state after the execution of a click(). However, these actions are wasteful because Selenium would throw an error if the action wasn’t successful. Furthermore, the next action would fail with an exception if it depended on the state of the previous element.

Potential waste: 2 seconds per element.click()

The refactor using cleaner practices

The refactoring of this test was extremely simple and required only a few minutes after understanding the root cause of the problem. Furthermore, we saw that the final outcome of the refactoring created a test that was 560% faster and drastically more stable because it had fewer Selenium commands.

Here’s the code after we finished refactoring:

Nikolay Selenium 11

This is the test execution result and below is the screenshot.

Nikolay Selenium 12

Lessons learned

Bloated tests in the industry are infamously common and it’s something that I tackle on a weekly basis. They typically come with a framework that we use for our automation. I’ve seen this with both OSS frameworks and especially homegrown ones. The key is to consider the impact of extra Selenium commands on the overall automation program. In this use case, the test suite execution time will be five times longer for the entire organization. This obviously decreases the feedback cycle time and also makes the automation less reliable.

 

Written by

Nikolay Advolodkin

Topics

Automated testingSelenium

Categories