Behavior-Driven Development is a software development methodology that promises to resolve the common communication issues that we face between business and technical people. Here’s a snippet from Cucumber documentation:
"Behavior-Driven Development (BDD) is a set of practices that aims to reduce some common wasteful activities in software development:
Rework caused by misunderstood or vague requirements
Technical debt caused by reluctance to refactor code
Slow feedback cycles caused by silos and hand-overs
BDD aims to narrow the communication gaps between team members, foster better understanding of the customer and promote continuous communication with real world examples."
Unfortunately, as is all too common in the tech industry, we spend a single hour learning about a topic, skip over all of the important documentation and recommendations, and turn the topic into a Frankenstein of our own making. As a result, we create concepts such as:
Fragile (Waterfall Agile)—where we take a few pieces that we like from the Agile manifesto and combine it with waterfall,
Selenium WebDriver for API automation—where we take an automation tool and start using it to solve all automation problems,
Custom implementations of page objects—where rather than keeping it simple, we create unnecessary abstractions, and
The opposite of BDD, BDD automation
As a Solutions Architect (SA) working for Sauce Labs, I get to see about a dozen clients and probably a hundred automation engineers a year. Hence, I see a lot of different code and implementation approaches. The one that consistently causes the most problems is BDD test automation. As a team of SAs, with a combined 100+ years of automation experience, we have seen one good implementation of Cucumber automation from Aslak Hellesøy, the creator of Cucumber.
In my experience, the biggest problem with BDD automation is that almost nobody follows the principles and ideas of BDD, as prescribed above. Instead, we simply hear a buzzword, BDD, and if we build it, we hope that it will solve our challenges. BDD is an excellent idea. The problem is that we don’t follow what has been laid out by the creators.
Yet it’s funny because although technical challenges can certainly hinder automation, cultural problems will make good automation impossible. “BDD aims to narrow the communication gaps between team members, foster better understanding of the customer and promote continuous communication with real-world examples.” (Cucumber.io)
I will never say it better than the creator of Cucumber himself:
"If you think Cucumber is a testing tool, please read on, because you are wrong.
There is a process to follow that involves many roles on the software team.
This process is called BDD. It’s what came out of that clique I mentioned. BDD is not a tool you can download."
--Aslak Hellesøy, The World’s most misunderstood collaboration tool, Cucumber.io
Let’s discuss the problems that we open ourselves up to when we start trying to do BDD test automation.
1. BDD automation is being performed by the wrong team
Who is creating the actual BDD spec files in your organization? Is it the three amigos sitting down together, talking, collaborating and leaving with a common language around the system requirements?
If that’s not the case, then that’s already going against BDD conventions.
However, in most organizations, the automation engineer is the one that is simply converting manual tests into Gherkin scenarios. The product persona refuses to learn the proper Gherkin syntax for test automation. Hence, the automation engineer is left to figure this out on their own.
If we think about the actual term Behavior- Driven Development, the behavior of the system should drive the design of the system. Who on the team knows the most about the behavior of the system?
Automation engineer
Manual tester
Business Analyst
Developer
You can pick any option except the automation engineer and they would all have more understanding about the system behavior!
So, why is the person that is least qualified actually writing the Behavior Driven automation scenarios?
Something isn’t right here…
And that’s the whole problem. It’s not that BDD is bad. It’s an excellent idea. It’s that we’re not doing BDD. And we’re certainly not doing BDD automation.
2. BDD Automation Tools Create Extra Dependencies and Abstractions
Let’s take a look at a diagram that shows the minimum amount of dependencies that are created when using a BDD automation framework (I didn’t include all the other dependencies such as test runners and so on, as they’re not related to BDD).
If you were to not use Cucumber or another BDD automation tool, your dependencies for testing would look like this.
In software development, we strive towards having our modules doing less while limiting the number of dependencies. Every dependency is a chance for something to go wrong. Hence, at the core of most design patterns is the idea that you want to limit dependencies, including Single Responsibility Principle, Open-Closed Principle, YAGNI and so on.
So why are we adding extra BDD dependencies and abstractions to our automation if we aren’t following the process that was prescribed by the BDD creators?
What purpose are we trying to achieve, to improve team collaboration or to use a tool because it’s the hot new thing?
3. BDD tools struggle with parallelization
I think that we can all agree that parallelization is mandatory in today’s automation environment. If you want to provide any kind of decent coverage in a reasonable amount of time, parallelization is mandatory (feel free to watch my presentation of parallelization).
Tools such as Cucumber and SpecFlow do parallelization in a sub-optimal manner. They parallelize at the feature file level. This implies that if you want to run 50 tests in parallel, you need to have 50 feature files. That’s a lot of feature files.
I have seen groups that spend company time and money to design a sharding mechanism that will split every test method into its own thread so that they can run scenarios in parallel, rather than feature files.
If you use a BDD automation tool, you automatically have a disadvantage in test automation. To overcome this disadvantage you must do extra work.
4. We are not actually writing correct Gherkin syntax
The argument that I hear most often for the usage of BDD automation is that it makes the tests more readable. And then I see tests like this:
(This is just a small and obfuscated sample of the BDD I see on a daily basis.)
Please describe to me what this test is actually validating from the end-user perspective?
The issue is that the majority of us do not follow the correct Gherkin syntax as prescribed by the BDD creators. I get it, it’s hard. For example, here are some of the rules:
Write all steps in third-person point of view
Write steps as a subject-predicate action phrase
Given-When-Then steps must appear in order and cannot repeat
So the point of having more readable tests as a result of BDD practices is actually erroneous.
Conclusions
Why are we, as an industry, using a tool for test automation that doesn’t provide any of the expected benefits, while at the same time creating more complexity?
As stated by the Cucumber documentation, “BDD aims to narrow the communication gaps between team members, foster better understanding of the customer and promote continuous communication with real world examples”. If we aren’t actually communicating more to promote a better understanding of the product, then it doesn’t make sense to add an automation tool that creates more complexity, hinders your ability to parallelize well, and overall just makes our life as automation engineers even harder.
