Anyone can chant, “We want quality, and we want it now!” But without a clear path to achieve it, you may find your team feeling stagnant or lacking the direction and focus needed to accomplish quality. You need a roadmap! So what goals can help you stay on the road to higher quality? There are many things you can consider, so let’s focus on a few key areas. I’m going to talk about things that we have deemed important to help us build in quality on our path to Continuous Delivery (and not just talk about why you should frontload quality).
Automation. (Period.)
We tried. Honestly. But a manual testing world is just not sustainable if you want to be Continuous anything. Since almost everything we had (not that long ago) was manual, sometimes we wouldn’t catch unintended impacts until DAYS later. In order to be successful, you need to look at automation. (If you haven’t read it, Greg Sypolt has a great post here on getting started.)
But what to automate? Of course more unit and integration tests are obvious, especially in keeping with the testing pyramid. Here are a few specific areas my team is looking at as a guide:
User Acceptance Tests (UAT) - Having a UAT helps verify key end-to-end workflow functionality of a feature. In my last ten years of testing, our automated UAT (which runs nightly) has been the key indicator of the health of our product. Instead of waiting days (or weeks) for the next manual run of a test script to ensure your primary workflows are still functioning as expected, you can run tests whenever you need to to check your work. You may want to also consider pre-commit tests that — you guessed it — run before you commit anything. You’ll know early if you broke something critical.
Rest APIs - Do you have tests for every endpoint? Great! But it’s not enough. Every method/parameter needs a test. Are your Agile teams aligned to have both backend and front end engineers? I hope so! This allows one team to own all aspects of a story.
Visual Regression Testing - The goal of this type of testing is to catch unintended visual impacts of styling and other changes. The basic idea is to use a framework to drive a browser to key points in your app and take screenshots before a change, and compare those with screenshots taken after a change.
Test Coverage - All of this is great, but how do you ensure you actually have code coverage? Leverage tools that can help you actually SEE what you have. (Jacoco, Clover, Coveralls are just SOME examples). Make a game of it! Can you beat your last percentage as you add new code or fix bugs?
Manual Testing is Still Important!
When setting our roadmap, we wanted to call it out as a continued goal in achieving high quality. Just be careful that your goals don’t begin to impede on automation. Manual testing should be a very small part in ensuring your product is amazing. (I previously touched on the importance of having a human perspective when testing in this post.) Make sure you have expectations well defined, or else it can easily get out of control. Things we are highlighting:
Engineers are testers, too! - Don’t start thinking that just because you have all this shiny new automation you don’t have to actually use what you just built. Engineers should develop the mindset that what they have built is ready for production BEFORE they even hand it off to QA. Use your feature in every way you can possibly think of, and if it makes sense, add more automated tests to it!
Exploratory Testing and Bug Bashes - These have been invaluable to us. But, as with anything, they can be overused. Have a clear direction, charters, and planned sessions. Be willing to acknowledge when something is losing its luster. Take a look at your frequency, incentives, and other factors. Adapt, learn, implement, iterate.
Browser and Device Coverage
Taking everything above into consideration, in a world where more people are using mobile devices over desktops, is your feature compatible with it all? With a myriad of devices to choose from, how do you pick? Part of your roadmap should consider this. Perhaps it makes more sense to utilize a vendor (ahem, hello Sauce Labs!)
Don’t forget the NFRs!
I’m talking about Non-Functional Requirements. Do you have clear goals for accessibility, localization, security and performance? Can you extend your automation framework to do some basic checks? Should you use a vendor? Each of these elements is still important in the quality of your product.
Summary
All of this is moot unless you are really frontloading quality. You cannot wait until the end to consider any of this in your testing. Make it part of your acceptance criteria. Build until you have met that criteria. When your tester gets it, all of these things should have been covered.
There may be bumps and potholes along the way, but I hope this information helps on your road to quality. Are there things you like to consider when setting your roadmap? I’d love to hear them!
Ashley Hunsberger is a Quality Architect at Blackboard, Inc. and co-founder of Quality Element. She’s passionate about making an impact in education and loves coaching team members in product and client-focused quality practices. Most recently, she has focused on test strategy implementation and training, development process efficiencies, and preaching Test Driven Development to anyone that will listen. In her downtime, she loves to travel, read, quilt, hike, and spend time with her family.
