Top 5 Quality Disasters (or Misses) of 2016

Posted Jan 6th, 2017

2016 was another year of questionable quality in almost every industry. Here are some defects (ranging from amusing to scary) that really caught my attention over the last year (some because they personally affected me!), and a few lessons we can learn from them as we develop our own test strategies such as data usage, environments, security, and more in our day-to-day work (please note these are simply my opinions, so please apply your own grain of salt!)

#1 - Social Media: Facebook Told My Friends I Died

There is nothing like getting a message from a friend out of the blue asking if I was OK... of course I am! Life is amazing! But to many of my friends, they saw a memorial on my Facebook page. While I got a little laugh out of it, the more human aspect of such a flaw is concerning - what if my grandmother had seen that?

The odd thing about this was, the memorial was visible for some, not for others (and definitely not myself! I didn't know I had died!), and it also wasn't consistently reproducible on desktop vs. mobile (isn't that one of our pet peeves as testers - not being able to reproduce something all the time?)

What can we learn from this? Let's say you're adding in a feature for an account to display someone's actual status on life. Always, always consider the end user. Consider the workflow to activate such a switch. What are the impacts if it is on and false? Easy to fix, but potentially a big ripple effect even if it's out there for a few minutes. Something I like to do for each feature I tackle is consider a 'headline' testing approach. If we build something, what could possibly go wrong, and what would make the news? If you have a feature that could mark someone as not alive, you really need to consider all workflows around getting to that point - not necessarily just primary workflows.

#2 - Communication: Skype Spam

Image Source:

I owe my 184 skype contacts an apology for the baidu links you received from my hacked account back in November. I’m sorry. I hope you didn't click on it. I did get nearly 150 reminders to change my password though. Message received. So what happened? While Microsoft indicates there did not appear to be a security breach, they did point to hackers are using username and passwords combinations obtained illegally (are they ever obtained legally?!) to gain access.1

What can we learn from this? Username and password is not enough these days. Apparently neither is two-factor authorization. And, people are lazy (myself included). I HATE changing passwords. Always consider the non-functional requirements (NFRs) - in this case: security. I can't say how many times I have seen the NFRs pushed out of a sprint and not included in the definition of done. These are CRITICAL to app success. For something as widely known as Skype, a quick Google search showed me blogs on how to hack into an account. No, I didn't try it (and I refuse to click on any links saying, "download this Russian hacking software now!") - but the information is out there. What can you do to stop it? Unfortunately there will always be hackers, but, painful as it is, change your passwords (more than once in a blue moon).

#3 - Devices: Samsung Galaxy note 7

Image Source:

Maybe I should just start a category called "Catching Fire" (Remember hoverboards from 2015?) Every flight I have taken this year, it was amusing to hear during the part about electronic devices, "and if you have a Samsung Galaxy Note 7, please alert the flight attendant. These are now banned by the Department of Transportation."

Essentially, as with most spontaneously combustible items, the Galaxy Note 7 came down to the lithium ion battery and the highly flammable liquid inside it.2

What can we learn from this? I don't have much to expand upon here from the hoverboard incident, but since this continuously happens, I think it's worth repeating:

While this may sound like a hardware issue, there are some software testing lessons here.

Anything that takes multiple parts to make means you not only need to test the sum of the parts, but the parts themselves. In my world, I'm talking third-party apps or plugins that enhance our product. Do you build a text editor from scratch? Or (most likely), did you decide not to re-invent the hoverboard wheel and use a third party editor like TinyMCE? But I bet you tested that third party plugin (or at least ensured that they did a lot of testing themselves).

Another takeaway from this - do mobile stress tests on the battery! I'll never forget one session when I was testing a feature on a mobile app, and by the end of the session, my phone was BURNING UP and my battery had almost completely drained (from a full charge) after just 20 minutes of using the app. (No, we did not release that version). Not only do you need to test your apps for functionality, but also how they impact your device!

#4 - Toys: The Unhatchable Hatchimals

Image Source:

Anyone remember the Cabbage Patch doll fever of the 80s? I think Hatchimals was this year's Cabbage Patch reincarnation. The elusive Hatchimals toys cost anywhere from $69 to over $200. My kids begged for one, but in light of reports that the toys didn't actually hatch, or took an incredibly long time to do so, I opted to pass on purchasing them.

What can we learn from this? Are you an early adopter of anything tech-related? Personally, I tend to pass on first-generation items -- whether it's an iOS upgrade, or a new phone -- and let the manufacturers sort out the big kinks before I adopt. But if, unlike me, you are an early adopter, you have to keep in mind that there will be some risks involved.

In the world of software, fixes are usually easy to "roll out" via patches (although it's still better for everyone if the issue can be avoided in the first place by testing before release). But in the toy industry, things are different. Recalls and irate customers who purchased toys such as Hatchimals could potentially sink a company.

To avoid that type of situation, one of my favorite things to do when performing exploratory testing is "Headline Testing." I ask my team what the worst thing is that they could wake up and read about our product in the newspaper. How would they address the issue? Take that thought-experiment and apply it to your testing approach when preparing a first-generation product for release.

#5 - Automobiles: Are We Ready For Self-Driving Cars?

Image Source:

This past year was an exciting one for the automotive industry. The dreams of my childhood watching the Jetsons are coming true: We have self driving cars! Of course, these cars are not without problems. Tesla in particular has been under intense scrutiny following a fatality in one of its self-driving cars.

What can we learn from this? Perhaps more than any other technology, self-driving cars highlight the importance of understanding the fact that users will not always make use of your product as you intend. Even if your technology is perfect, you need to factor human error into the equation when you test it.

How do you plan for human error? Testers can help by making sure that they run tests not only for primary and secondary workflows, but also for modes of operation that product designers might never have imagined. This is a situation where testing on real users under real-life conditions can come in handy.

The year in review

What were your favorite quality disasters of the year? I look forward to what 2017 will bring!

Why are Skype accounts getting hacked so easily? Tom Warren. Nov 8, 2016.
Here's why Samsung Note 7 phones are catching fire. Sean Hollister. Oct 10, 2016.

Ashley Hunsberger is a Quality Architect at Blackboard, Inc. and co-founder of Quality Element. She's passionate about making an impact in education and loves coaching team members in product and client-focused quality practices. Most recently, she has focused on test strategy implementation and training, development process efficiencies, and preaching Test Driven Development to anyone that will listen. In her downtime, she loves to travel, read, quilt, hike, and spend time with her family.

Written by

Ashley Hunsberger


Secret Sauce