Software testing is evolving, and model-based testing is an integral piece of modern test automation. This blog post explains what model-based testing is all about, discusses the challenges and advantages of model-based testing, and offers inspiration for a kick-start in the world of generating test models.
What is Model-Based Testing (MBT)?
Using Selenium in itself is a challenging and daunting task. Model-based testing is a testing approach where test cases are automatically generated from models. The models are the expected behavior of the system under test and can be used to represent the testing strategy. A model is an abstraction of the real-world function. From the QA perspective, we strategize what to test since we are unable to check everything. MBT allows us to share the responsibility with software developers and testers to only focus on the models that need to be created around system requirements/specifications.
How does it work?
It generates tests automatically from models created by software developers and testers. The test oracle runs assertions for both generation of tests and execution. The test executor runs all the generated test cases and reports the testing results from the model's abstract test suites.
What is a test oracle? (That was my response the first time I heard this terminology). In computing, software testers and software engineers can use an oracle as a mechanism for determining whether a test has passed or failed. The use of oracles involves comparing the output(s) of the system under test, for a given test-case input, to the output(s) that the oracle determines that product should have.
Let's take a closer look at an actual model for simple login with valid credentials:
How is Model-Based Testing Different from Other Forms of Testing?
With an upfront investment into making a more reliable and maintainable testing suite for a given application, MBT is more part of the software development process than independent scripting tasks. The team focuses on how to build a testable application and create models based on real-world functions from the user perspective. No more test script development and no need for test script maintenance.
Hold on a second!
MBT doesn't eliminate maintenance; the models are created and maintained within the code and are part of the software development process.
How is this going to help software developers and testers? It helps create better software quality by getting the team thinking about the models. It reduces test suite maintenance. It allows for flexibility to generate many tests using different algorithms (smoke, regression, integration, end-to-end, and targeted testing for new/modified features).
The Challenges of Model-Based Testing
The move towards models holds tremendous promise, but it also introduces new challenges that must be addressed before you can receive the full benefits of MBT. The work starts by training the software developers and testers on model-based testing. The learning curve is steep. While software developers are familiar with coding paradigms, they still require testing knowledge, and they still need to know how to develop testable applications. Testers will need to learn a new concept of modeling over traditional testing methods.
Before diving into MBT head-first, invest in the infrastructure. Modeling is hard. Take the time to think about architecture. The tool must be scalable, and it should handle complex models and provide reliable test coverage. It will require more significant upfront time for the initial implementation of the tool. It will offset the long-term goal of reduction of test maintenance and increased test coverage. Fine-tuning the MBT tool in parallel with the creation of models could be challenging and can result in refactoring of the tool.
The adoption of MBT starts by implementing modeling into the development workflow. It might be a challenge, since it requires a shift in mindset and culture as far as how to develop and test applications. It's crucial to make modifications to frontend application code to improve the testability—fixing found bugs, while creating models for the system under test.
The Benefits of Models
It starts with specifications by reinforcing the idea that QA involvement belongs at the beginning of the discovery stage. It forces testability into the product design when talking about the creation of models (the model is the test plan) for a new/modified feature. It typically finds design and specification bugs before the code even exists. The automatic test suite generation will increase testing thoroughness, test coverage is guaranteed, and there is zero test suite maintenance. It's part of your continuous testing strategy and reporting results to individuals or the team.
Conclusion
The purpose of this blog post is to provide the necessary understanding of what model-based testing is, and the challenges and benefits of shifting to models.
To learn more about MBT solutions, here are a couple of talks from the industry: Spotify Model-Based Testing by Kristian Karl, Model-Based Testing: A Practical Example by Neil Sander Christensen, and coming soon, my team will be conducting a Workshop on Model-based Testing at STPCon Spring 2018.
Greg Sypolt (@gregsypolt) is Director of Quality Engineering at Gannett | USA Today Network, Fixate IO Contributor, and co-founder of Quality Element. Responsible for test automation solutions, test coverage (from unit to end-to-end), and continuous integration across all Gannett | USA Today Network products, he has helped change the testing approach from manual to automated testing across several products at Gannett | USA Today Network. To determine improvements and testing gaps, he conducted a face-to-face interview survey process to understand all the product development and deployment processes, testing strategies, tooling, and interactive in-house training programs.
