QA is Not Enough: You Need to Engineer Productivity

If you stay at a company long enough, you see a lot of changes in leadership and organizational “restructuring.” Most of the time, you just ride it out with little impact in your day-to-day work, and before you know it—the next wave has come in, and things shift again. (And if you have really been there a long time, you see the same patterns repeating.)

Until recently, I had yet to see a shift that was truly impactful to me. Now, my team is no longer conceived as a QA team, but rather as an Engineering Productivity (EP) team. The shift was about more than just a change in titles.

Below, I’ll explain what the shift to EP means—both at the superficial level and in terms of the actual work we do.

As Patrick Copeland describes in How Google Tests Software, “Productivity is our job; testing and quality are the job of everyone involved in development. This means that developers own testing and developers own quality. The productivity team is responsible for enabling development to nail those two things.”

When we made the name change from QA to EP, we wanted to establish what our mission was. Put simply:

Reduce the time from concept to deliverable by providing our product development teams with the tools, practices and support to increase their productivity while maintaining high quality standards

Notice that nowhere above does it say “we are the testers.” To meet our mission, we established four major goals for our projects that we continue to drive towards:

Goal #1: Provide an easily maintainable and extensible framework that enables scrum teams to add and remove tests. This is not just the what, but the how. What are the strategies and guidelines? How do we decide to include tests? How do we maintain them? Are teams clear on our vision of testing? If teams don’t understand, how can they be successful?

Goal #2: Enable the automatic and early detection of failures within the software under development. I always talk about failing fast and a “shift left” mentality for quality (testing as early as possible). We all should know by now the cost savings of finding a defect earlier rather than later (thousands). By enabling teams to do this more easily, the engineers get faster feedback on their code. When we first started, we were running tests at merge. Now, we will be running at every pull request.

Goal #3: Prevent the source of detected failures from moving any further downstream. It’s not enough to just detect the failure—You have to prevent it from making it to production. We work very closely with the CI team to ensure our gates are in place. It should be no surprise if we do not distribute a build if our tests failed. Teams start to see the importance of being in a releasable state, and ensuring their code changes result in passing tests.

Goal #4: Accommodate all of this without impacting the engineers’ time. This is not easy work, but we want to provide everything without really impacting the engineer. It doesn’t mean that an engineer is not involved in the testing—It simply means we help them adopt the culture of delivering with high quality, which becomes ingrained in the entire Scrum team.

These probably sound all too familiar, and almost no-brainers. But if I’m being honest—I don’t remember the last time someone actually laid out these goals for me point-blank, even though we’ve been working very much towards something like this. With clear goals established, the map was laid out for us.

Once we knew our mission and goals, we needed to ensure we knew our roles within the team. The Engineering Productivity group is made up of Test Engineers, Software Engineers in Test, CI/DevOps Engineers, and Engineers.

Our Test Engineers (TEs) drive our test strategy and help identify what needs to be tested. I like to think of them as the facilitators. The TEs tend to align with a feature Scrum team. Software Engineers in Test (SETs) build the frameworks that enable teams to implement the tests. While some may be embedded on Scrum teams to implement tests, we also have core teams to establish and maintain the health of our frameworks. Our CI engineers own the pipeline and help implement the work to make improvements as we identify them. While they are their own team, we do embed them within the EP teams responsible for the framework and pipeline. Another role that has been crucial to teams buying into our mission is the developer—in our case, one that understands engineering requirements, and is passionate about meeting those needs. We are so fortunate to have found an existing engineering team member who wants to be a part of the solutions we are looking to provide. (If you find that person, never let them go!)

My end game, though I know this may take awhile to achieve, is to become invisible. There will always be a learning curve as we roll out new ideas to further engineer productivity, but my goal is that each of these ideas become as second nature to engineers as breathing.

Ashley Hunsberger is a Quality Architect at Blackboard, Inc. and co-founder of Quality Element. She’s passionate about making an impact in education and loves coaching team members in product and client-focused quality practices. Most recently, she has focused on test strategy implementation and training, development process efficiencies, and preaching Test Driven Development to anyone that will listen. In her downtime, she loves to travel, read, quilt, hike, and spend time with her family.