Every few years, new development and release methods appear. Lately, most of us are looking at continuous integration and continuous delivery. But what about those teams that are trying to transition to faster releases? That’s going to be a huge hurdle if you save your testing for the end, before release (and in my experience, that has almost always resulted in pushing the release date). There is an important shift that needs to be made in order to be successful: bring in QA early.
Moving Upstream - From the Field
I recently participated in a sprint retrospective. During our ‘lessons learned’ meeting, an engineer said, “I really wish we had had these conversations with QA earlier
... they really knew the feature.” In my perfect world, the referenced discussion would have happened before any development had begun. Unfortunately, sometimes factors are out of our control (like the number of resources — I have discovered that I cannot, in fact, clone myself). The end result of this meeting was that the engineer realized that some unexpected new work and some revisions to code he had already written would need to be done. Extra scope aside, the engineer brought up a great point: the tester knows the system/feature, and really does know what questions to ask.
When you bring QA in earlier and discuss a feature up front with the engineer and designer — before any work is done
— all questions and assumptions are out on the table. This pre-mortem allows the team to size up possible issues, and prepare for feature testing with better expectations. Together, you eliminate the guesswork that comes from working on your own, which will likely be different depending on whether you are a developer or a tester. When QA and devs work together, engineers know precisely what they have to develop in order for the feature to pass and meet the acceptance criteria, and when it is time to audit, designers should not be surprised.
The benefits of moving QA upstream go beyond getting everyone on the same page. You will also reduce the overhead of over-documenting your project, since the acceptance tests become your specifications. There is no more need to tediously update both test and spec if you are defining them in your acceptance criteria as a team. I wish I had a dime for every time I heard, “Was it updated in the spec so I can update the test (or update the code)?” …
Let’s just say I’d have a lot of dimes. Often this conversation happens in various emails or chats where someone thought the behavior was discussed, and may or may not have included all necessary team members.
To me, the biggest reward for defining acceptance criteria first is that you are building quality in from the start.
Of course you’ll find bugs — I wouldn’t be where I am today if software came bug free. But you will certainly prevent bugs, and hopefully the bigger ones. Think about it, if you define acceptance criteria first, you build to the test until all tests pass — the bugs found during that process are found early so you can release faster but maintain quality. The cost alone for finding a bug late is so much higher than finding a bug early (spending resources on triage, remembering what was built a sprint or so ago while you’ve possibly moved on to another user story, potentially impacting other code with your fix).
Step 1 - A Strategy
To be truly successful with continuous integration, I think you really do need to look at automation — but the great thing is, moving QA up in your development lifecycle is completely tool and methodology agnostic. What do I mean? It doesn’t matter if you have a manual testing group or a robust automation shop. Getting your QA involved earlier can only help your team. Of course, as you write your acceptance criteria, you may want to also identify how that criteria can be tested (for example, is it a unit, integration, or functional test), but in the grand scheme of things, it doesn’t matter what your testing framework is. Hopefully you quickly identify very little that needs manual testing to free you up to explore the system in other ways.
Naturally, there will be some skeptics. So QA also needs to educate the team in the value of this approach, and why it’s so important that everyone view quality as a team effort. Just because we may help drive the discussion and define acceptance criteria, that doesn’t mean the tester owns the quality. Engineers may need to write more tests than they are used to, but it will only help their code as they build until every test passes. With everyone working together, engineers, QA, and designers all take part in defining quality, owning it, and delivering a better product.
Ashley Hunsberger is a Quality Architect at Blackboard, Inc. and co-founder of Quality Element. She’s passionate about making an impact in education and loves coaching team members in product and client-focused quality practices. Most recently, she has focused on test strategy implementation and training, development process efficiencies, and preaching Test Driven Development to anyone that will listen. In her downtime, she loves to travel, read, quilt, hike, and spend time with her family.