Why Manual Testing Helps Your Release

Will we ever truly be at 100% automation?  I hope not. Of course automation is critical in implementing Continuous Integration and Delivery, but there are just some things that you can't leave to a machine. Human evaluation is important. In a world where we are looking to release faster and faster, why would we want manual testing?  Let's take a look at some of the things you may want to do that automation can't, and how manual evaluation helps us deliver the right product.

Several years ago, our UX team kept asking, "Is it delightful?"  I’ve worked on many features that I truly felt would make for a better experience in education.  There are several that, frankly, I just couldn’t stand. We just weren't building the right product sometimes; even if all tests passed, and there were no bugs — if I didn't like using it, I found myself asking, "How would users feel?" I have to say, I'm fascinated by the human factor and evoking feelings (for better or worse) when testing software. As a consumer of software, sometimes I find myself thinking, Wow, did ANYONE look at this? (For example, I'm on the Board of Directors for my Home Owner's Association, and the software we use to track documents, get assessments, and so forth just makes me want to cry). I'll be honest - sometimes it is difficult to see how usable something is until there is something to use it for. Hopefully, though, we can spot this early as we define acceptance criteria and are evaluating the workflows, specs, wireframes, or prototypes.

The obvious manual testing activity that should be at the top of everyone's list is exploratory testing. In an ideal world, the features themselves are completely automated, and development is done when all tests pass. This is fantastic, but what if that were it?  Lewis and Clark had specific goals, and reported what they found along the way. Exploratory testing is similar to me: I start with a charter (or a goal) from a user perspective, and report what I find along the way. Perhaps it is a feature that works fine in unit and integration tests, but once I start looking at it myself in an end to end workflow, I think of other scenarios that perhaps we didn't think of when writing scripts.

One of the biggest successes I've seen lately is with a bug bash.  This activity is open to people even outside of our dev teams. As features complete development and pass initial rounds of testing, we open them up to a bug bash. People from QA, Engineering, UX, Product Management, Support, and beyond have been involved, getting a wide range of perspectives and scenarios that perhaps were not thought of before. Use exploratory testing to your advantage — there is no one way to do it. Do what works best for you! Get different perspectives, think like the user, and see how the product makes you feel.

Another area of testing that I'm very passionate about is Accessibility. Not everyone uses the Web in the same way.  (To see some of the many ways in which someone may be hindered, read more in this fascinating post: https://the-pastry-box-project.net/anne-gibson/2014-july-31). Yes, there are tools and scans that can be used to check for standards. But if I don't check for accessibility manually, how can I experience whether keyboard navigation occurs in a logical order when tabbing through, or know (by seeing it in person) that what the screen reader calls out makes sense? To celebrate Global Accessibility Awareness Day, we recently held a bug bash where we asked users not to use a mouse. While most features held up well to the challenge, it gave us more perspective on trying to build the right product, and building one that is delightful to use. (To learn more, read: http://blog.blackboard.com/mouse-free-an-accessibility-challenge/).

Consider that we live in a world where most people are accessing your software on their mobile devices.  Now consider what you look for in an app. Personally, there have been moments where I've tried out five or six running apps — only to quickly delete most of them after about ONE MINUTE if they aren't user friendly or don't meet my needs. ONE MINUTE. If you don't take time to run through your app and explore how it is to use, why would your clients? If I can't figure out how to do something quickly, I'm pretty much done with that app. DELETE. (Don't let that be you.)

I hope it's pretty clear that manual testing can definitely help you deliver the right product to users, and a fun one at that. Of course you don't want to save manual testing for too late (the costs keep rising the later issues are reported), so try and find a happy balance from your feature/dev branches to the test environment, and you will start seeing the usability much earlier. Although Continuous Delivery provides for a constant feedback cycle and opportunity to learn and fix issues quickly based on that feedback, use manual testing to your advantage and make clients happy from that first (or thousandth) release!

Ashley Hunsberger is a Quality Architect at Blackboard, Inc. and co-founder of Quality Element. She’s passionate about making an impact in education and loves coaching team members in product and client-focused quality practices.  Most recently, she has focused on test strategy implementation and training, development process efficiencies, and preaching Test Driven Development to anyone that will listen.  In her downtime, she loves to travel, read, quilt, hike, and spend time with her family.

T: @aahunsberger L: https://www.linkedin.com/in/ashleyhunsberger