Introduction to Usability Testing

Do you ever find yourself wondering if you are building the right product? (Or worse, not thinking about it?) Despite best intentions with Agile, it is easy to lose focus on the client and become ingrained in the technology you are building. But how do you get back to center? Usability Testing is a great way to refocus on the customer. Not only will it put you on the path to the right product, but it can shape how you test moving forward. Simply put, usability testing lets you see just how easy your application is to use.

I'll never forget my first usability test. Our team had worked for months (probably the better part of a year) on a feature we wanted to release. We felt we were in pretty good shape. We sussed out the major bugs, and were feeling pretty confident. When we started the test, it became apparent just how unusable the feature was. It was scrapped. That can be pretty demoralizing.

A good usability test session starts with a good script, and knowing your players/roles.

A good script should have some clear, achievable goals, without explicitly telling a user how to accomplish a given task.

The facilitator serves as a guide, using a script to help steer a client through a desired set of actions. Ask leading questions if the user gets blocked or frustrated. Elicit emotional responses. Ask how the user felt. Almost all of these things cannot be told by automation (other than how to complete a task).

The recorder takes notes. If the test must be virtual, capture every murmur, comment, sigh, and the length of time it takes to complete an action. I've served in this role before, and let me tell you - clients don't have a filter. If you are able to conduct the testing in person, you capture the scrunched eyebrows. You can follow their eyes. Did they go straight to the feature? Overall, you look to record whether the user was able to achieve the tasks in the script, and the ease (or lack thereof) with which they did so.

Of course, this testing is senseless without the end user! Don't just think about one end user. At Blackboard, we have held sessions for instructors and students separately. Your script should vary depending on the audience. (You probably don't need to run a session with a student for an administrator workflow, for example.)

The hardest thing is to not tell your end user how to specifically do something! While I pride myself on thinking like the client, I do find that some tasks become second nature (especially after working on a particular feature for so long), and as a result I take for granted that maybe a workflow isn't so obvious.

Who can make good candidates to take on the roles of facilitator or recorder? With some training, this is a great opportunity for various members of a scrum team to talk directly with clients! Many times you are siloed, relying only upon the PO and designer to represent the client. Hearing a client first-hand is a good way for the scrum team to truly understand their needs.

Jakob Nielsen, a leading usability expert, developed usability testing heuristics. Some of my favorites include:

So why is this useful (beyond the obvious of finding out if you are building the right app)?

Nielsen has published research that has shown that conducting usability tests with 5-8 people can identify ~85% of usability problems. By understanding those problems, you can use that data to identify patterns and understand where you may need to invest a little more in your design and test strategy moving forward. Our automation is only as good as we tell it to be, but product usability can be determined when the team knows what to look for.

Do you know what else I found out through usability testing? Clients love to feel involved, and they love talking to the people that make what they use. It's more intangible, but building rapport with the client has made me more invested in building something awesome for someone I know.

Of course there are challenges. The main challenge is when to do this type of testing. Too soon could mean you are showing the client something without much value yet (though hopefully you are building value in with each story). And too late could mean you are too far gone to change paths if you don't get very favorable feedback (short of scrapping a project all together).

You should also consider the type of user. Some clients are power users, and some are novices. You will need to be able to adjust and still make for a meaningful usability session. Some may need more prodding, while others seem to be as adept (or even more adept) at using a feature as the developers themselves.

We all hear about building the right product, but not many teams are aware of this type of testing. Find out how usable your product is, and delight your users! This has definitely been one of the catalysts in my career, and it has shifted the way I think about testing.

Ashley Hunsberger is a Quality Architect at Blackboard, Inc. and co-founder of Quality Element. She's passionate about making an impact in education and loves coaching team members in product and client-focused quality practices. Most recently, she has focused on test strategy implementation and training, development process efficiencies, and preaching Test Driven Development to anyone that will listen. In her downtime, she loves to travel, read, quilt, hike, and spend time with her family.