Test script maintenance does not top many people’s lists of what makes a good time. Fortunately, however, maintaining software test scripts does not need to be a headache. While I can’t promise you that I’ll turn script maintenance into something you’ll wake up feeling excited about, I can offer tips here for making the maintenance process faster and less tedious.
Ways to Reduce the Maintenance Nightmare
How much thought went into your rollout of a front-end automated solution? Of course, like any new experience, I jumped into the deep end when I started the automation journey about nine years ago. I didn't know the best way to approach the transition to automation, but I was told to automate all the things right off the bat. (Oh-so-many lessons were learned from that initial experience.)
Who really owns test script maintenance? In my opinion, the developers own quality, development of tests, and maintenance. It all depends on the culture of the company. Quality is a team effort. I work at a large company (Gannett), and we have a team dedicated to helping the development squads take control of testing to ensure success. The dedicated quality engineering team is available to individuals and the development squads to help deliver stable, quality products. The quality engineering team values reliable, efficient, and effective tests that provide the coverage necessary for each software product. It doesn't mean making as many tests as possible — it means targeting tests that provide the most value for your product.
The shift to quality engineering has allowed us to spend the right amount of time to build a stable testing platform, and to set standards for all testing layers to allow development squads to attain quality ownership. We're driving the strategy of what to automate, communicating the process, and choosing the best testing solution for the product.
The keys to low maintenance are enforcing standards and using design patterns to improve automated testing by educating everyone.
Patterns
We see design patterns everywhere like traffic patterns: stop sign, traffic signal, roundabout, and flyovers. The objective is to keep all directions of traffic moving and reduce congestion. To evolve, we need to continually evaluate and identify problems in the current design to determine how to keep traffic flowing and reduce congestion. The same thing applies to automation development. Why are design patterns so important for automation development? Because they can save time, improve reliability, speed up development and execution, reduce frustration, and reduce maintenance impact.
Traffic Pattern | Compare with Test Automation |
Stop sign works, but can’t handle high volumes of traffic. | You're getting started with test automation and learning the basics, not thinking about scale. |
Traffic signal improves the traffic flow but multiple lights on the route are out of sync and could cause congestion. | Introduce reusable components, setting standards around waits, and execute tests in parallel. |
Roundabout — designed to keep all four directions of traffic flowing, until that one person does not understand the rules and causes a massive traffic jam. | Without clear guidelines, people will get confused and cause traffic jams. Take the time to communicate your automated testing standards. |
Flyovers are designed for scale and high volumes of traffic. | Using cloud solutions like Sauce Labs to spin up as many virtual machines as possible to handle the volume of testing. |
The moral of the story: using design patterns in test automation development is not necessary, but a seasoned automation engineer understands the importance. If you are new to test automation, I highly recommend learning the best way to write automated tests and design for scale, which will reduce test script maintenance.
Don't Repeat Yourself
This principle is so important to understand: eliminate the need to duplicate code anywhere within your suite of tests. It may seem obvious to someone with experience, but it's often an oversight at the beginner level. Why would anyone want to copy and paste code? The duplication of code is a surprisingly common problem within software and automation development. It mainly comes from beginning-level programmers.
Why is code duplication so bad? It’s simple: duplication causes a maintenance nightmare whenever you need to update test scripts. Refactor all duplicate code by following the page object model. The purpose of reusable code is to encapsulate a web page entirely in a single file. It merely helps Selenium code to be more readable, maintainable, and reusable.
Waits
Defining a clear waiting for elements strategy, in my opinion, is seriously essential. Take the time to understand the wait strategies from sleep, implicit, and explicit. Please don't use sleep methods! (It's just going to cause you to pull out your hair.) If needed, use an explicit wait.
Locator Strategy
The most efficient way (and preferred way) to locate an element on a web page is to use unique IDs. When there is no ID to use, the next preferred choices are NAME and CSS locators. The only problem with using NAME and CSS locators is that sometimes a page may contain multiple elements with the same name.Why is this a problem? It’s a problem because Selenium is designed to find the first matching element on the page. To find the second or third element on the page, many need to use one of the other types of locators, like attributes. XPATH locator should NEVER be used to find an element on web pages or mobile applications. When developers own test development, they recognize the importance of coding unique IDs to the product.
Focus on Automating the Right Things, Not Everything
Minimize the number of UI tests (because they're fragile). Learn how to work with development squads to push the coverage down the stack to low levels since the developers understand the application code the best. Advocate keeping tests lean, modular, and independent. This will reduce test script maintenance.
Flavor Characteristics of Test Script Maintenance
It starts by recognizing some of the different flavor characteristics that could cause unreliability of test scripts — which result in maintenance.
Unstable environments - unable to handle high volumes of tests running in parallel; application infrastructure not repeatable and disposable, or environment not sized correctly.
Too many network hops - causing timeouts or unknown issues.
The web application isn't designed for testing. Developers don’t understand the locator strategy.
Poorly written test scripts - no coding standards or design patterns.
Data collision - no unique data or settings.
UI testing everything.
Lack of communication on software changes.
No quality ownership from developers.
Closing
Build a culture where quality is a team effort. Spend the right amount of time planning, and make a solid decision the first time so you won't need to refactor over and over. Don't be afraid to make mistakes. They're going to happen. Learn from them to reduce the maintenance burden going forward.
Greg Sypolt (@gregsypolt) is Director of Quality Engineering at Gannett | USA Today Network, a Fixate IO Contributor, and co-founder of Quality Element. He is responsible for test automation solutions, test coverage (from unit to end-to-end), and continuous integration across all Gannett | USA Today Network products, and has helped change the testing approach from manual to automated testing across several products at Gannett | USA Today Network. To determine improvements and testing gaps, he conducted a face-to-face interview survey process to understand all product development and deployment processes, testing strategies, tooling, and interactive in-house training programs.
