Searching for User Interface test automation on Google will yield a lot of results for tools, but not much in the way of good practice. Search for experiences with GUI test automation and you may get some code (in the wrong programming language) or you get some … advice. The advice is lately to include a handful of UI automation stereotypes -- UI automation is unstable, UI automation is slow, record/playback is brittle, teams will spend more time on maintaining test code than they will actually
This article covers some important aspects of automated website testing -- strengths and weaknesses, building tests, framework selection, and how automation fits into a software development strategy.
The first step, before taking a look at the many options for tools and before building proofs of concept, is to ask the question 'why'.
Arguably the most common reason companies start a UI automation project is because “regression testing takes too long.” Each release, they have a week of development on new features, then
The end result of this is usually a halfway build
That is the problem to fix.
Here is an alternate strategy for UI automation; use it to propel features forward into production faster.
UI automation is typically developed on a lag of at least one sprint. The test code
In-sprint automation looks something like this. Developers write their new features sprinkling in unit and API testing where appropriate. Once the UI is close to done, they should be working more closely with a tester. That tester can begin building the bones of an automated UI test. They might write a couple of lines of code to navigate to a page, and start populating some text fields and have to stop to investigate a field that isn't accepting an input 20 characters long. The developer would take a look at that point, fix a bug, and restart their local server. Then the tester would get back to building the test in that same write a line or two, stop to investigate something interesting and then start again.
By the time the important coverage for that feature is built, the new code change has been explored and had rounds of bug fixing. When the change is ready to be committed, there are unit tests, API tests, and a handful of automated UI tests. The feature is basically ready for production. This also has the side effect of preventing too much UI automation. At some point, the pair can decide that they have explored enough and have just enough automated coverage that it is time to move on to the next task. As the consultant, Matthew Heusser famously quipped “The story is not done until the automated tests [for it] run.”
Once the process in place, regression testing really is faster and development flow is a little more smooth. Instead of having to have a hand-off between groups,
This strategy and approach are crucial for teams that want to deliver software in a cadence that approaches Continuous Delivery.
There are a number of APIs and tools available to help people build UI automation. One easy way to categorize them is whether the person building tests records steps they perform in a browser and later adds assertions, or if they are writing code.
Record and Playback
Telerik Test Studio and HP Unified Functional Testing (UFT) are
Any of the tools mentioned above are fine, a team can end up with useful test automation regardless of whether they choose the record and playback tool, or the programming API. Which you choose will depend on the staff available. Record and playback tools work well with non-technical testers. Though, it is important to remember that no matter how much a tool claims to be code-free, some programming will eventually be required. Also, having a deep understanding of both testing, and automation is still required.
The coded API options, such as Selenium WebDriver, offer far more flexibility and are generally easier to hook into Continuous Integration
The first step to build a test for a user administration page using WebDriver would be to build the Page Object. This class should contain methods such as
The first test for that user administration page might look like this:
Those steps would be followed by assertions based on what you are expecting the page to return.
There is a fine line in automated UI testing between a shallow test suite that doesn't help you learn about quality in the important parts of your product, and suites that are overkill and cross that threshold of requiring constant care and feeding. (The more checking you do, the more maintenance the test will need.) One single user creation test for an administration page probably isn't enough. Automating every test idea that enters your head is probably too much. An approach in the middle, Create Update, Delete (CRUD), might make more sense.
This test would do something like the following:
- Create a new user
- Assert that the new user persists and values are as expected
- Update each updatable field on that user profile
- Assert that new values persist
- Delete the user record
- Assert that that user can not be found in search
This sounds like a complex scenario for web testing, and it potentially is. Good architecture and the use of design patterns like the Page Object can make this both useful and reliable.
If there were a downfall to the Page Object, it would be that a lot of tests will fail when an element on the page changes. That problem is true regardless of which development pattern is used. The strength of the Page Object pattern is that the fix only needs to go in only one place -- the page object -- instead of every test. The combination of the right design
There are as many ways to get UI automation right as there are ways to mess it up. Start by understanding the team’s goals: what problems is the development team having and how can UI automation smooth those. Next, build a strategy to fit those goals: how much UI automation is needed, what places need coverage, and where should the team start first. Then, select the right tool, a design pattern that works with