Blog
1 Simple Trick To Massively Improve Automation Efficiency
In this article, Nikolay Advolodkin presents a case study to show how interactions with only two web elements led to a 34% degradation in test execution time. He will show you how to remove this inefficiency by simply keeping the element location timeout to a maximum of 20 seconds on mobile devices.
Automated UI testing is a daily struggle for efficiency and reliability. A single misconfigured line of code can cost teams in hours of lost feedback time and test error triaging—potentially costing companies hundreds of thousands of dollars.
In this case study we will see how interactions with only two web elements led to a 34% degradation in the test execution time. Afterwards, we will learn how to remove this inefficiency by simply keeping our element location timeout to a maximum of 20 seconds on mobile devices.
What Our Final Test Can Look Like
Screenshot of the test metadata before optimizations
Trying to Find Element 1
In the screenshot below, we start looking for element 5015 at ~0:55.
And the test continues to look for this element until 02:12, at which point the test decides to move on.
What Were These Commands Doing?
Looking at the locator, screenshots, and video, it’s obvious that the test was waiting for a spinner to disappear. After a closer analysis, it’s clear that the spinner is gone and the mobile app is in a ready state by 1:00.
Afterwards, the test wastes 72 seconds waiting for a spinner to not reappear before proceeding with the next actions.
Inefficiency Cost
19% = 72 / 377, trying to find a single element.
What’s The Solution?
First, simply decrease the amount of time that we wait for elements to disappear. 20 seconds per element on a mobile device is sufficient. If an app isn’t rendering in 20 seconds on a mobile device, we have a bigger problem. 🙊
Second, the logic to wait for a spinner to disappear is actually incorrect. The test should wait while the spinner is visible. When the spinner disappears, at that point the test can proceed. As a result, the test will only wait as long as necessary for the spinner to go away.
Trying to Find Element 2
There is another inefficient pattern in this test that’s dealing with element 5033. At 2:23 the test locates this element.
After the test finds element 5033, it begins a cycle of calling the same three Selenium commands from 2:24.
This cycle repeats until 3:40 (76 seconds total), at which point a screenshot is captured and then the test is stopped.
What Were These Commands Doing?
These commands were looking for some element that never appeared.
The Problem
The extremely common issue here is that there is no reason to waste 76 seconds looking for a single element. This is simply too long. As mentioned previously, 20 seconds on a mobile device is sufficient for 99% of cases for a page to render.
Inefficiency Cost
15% = 56/377, wasted on looking for a single element.
What’s The Solution?
The solution is the same as above: decrease the timeout to 20 seconds maximum. What benefit does the test get if it spends 56 more seconds looking for an element?
Total Damage From Inefficiency
128 seconds out of 377 (34%) are spent on trying to locate only two elements! What’s even scarier is that this pattern is almost certainly applied throughout the entire organization as a result of the framework, likely costing teams dozens of hours every single day as a result of overexaggerated timeout values.
The solution to this problem is elegantly simple. Set a 20 second timeout for all your element locators on a mobile application. For the large majority of cases this will be more than plenty. If we find that this is not sufficient and we keep seeing false positives, then we can increase the timeout.
If you're interested in more like this, check out my other post on how to optimize a test and make it 560% faster.
