Top 5 Quality Disasters (or Misses) of 2015

2015 was quite the year for quality in almost every industry. Here are some defects (some disastrous, some just funny) that really caught my attention over the last year, and a few lessons we can learn from them as we develop our own test strategies such as data usage, environments, security, and more in our day-to-day work.

#1 - Social Media: Facebook tells me I’ve known you since before I was born!

Image Source: http://ti.me/1mp25FH Image Source: http://ti.me/1mp25FH

This was probably one of the funnier, and definitely not critical, bugs of the year. (Let’s face it — no one actually got hurt, and I think we all chuckled a little when we saw it pop up in our news feeds!) While no one actually _confirmed_ what happened, most theories surround the Unix Epoch (January 1, 1970, which with time zone adjustments could be sometime on December 31, 1969), so 46 years from the day we started seeing our nearly golden friendships appear. Microsoft engineer Mark Davis offered his hypothesis (see http://ti.me/1mp25FH).

What can we learn from this? Let’s say you’re adding in a feature that has to deal with dates. Always consider testing not just with fresh data, but also data that was present prior to the feature being added. I must admit that in my own testing we hate dealing with date/time stamps as it usually involves manipulating data in unrealistic scenarios, but test we must. Make sure you are doing some time travel with your data — it helps get you closer to what your customers will have!

We can also learn through this example why Continuous Integration and Continuous Delivery is so beneficial. How long did you see that issue around? Honestly I don’t remember seeing it very long, just maybe once or twice when I looked at Facebook. Since they practice CI/CD, they were able to get a fix out rapidly, versus waiting over long development and testing cycles to get a patch out.

#2 - The Federal Government: OPM Data Breach

Image Source: http://bit.ly/1Fw1vHe Image Source: http://bit.ly/1Fw1vHe

You probably thought I was going to talk about healthcare.gov … That was so 2013. But no, I’m going with the Office of Personnel Management (OPM). Have you had a government background check in the last 15 years? You probably got a letter like I did. Not only is your information at stake, but also information about anyone mentioned in any interview. 22.1 million people. 5.6 million fingerprints (after an initial estimate of 1.1 million).((Washington Post, ‘Hacks of OPM databases compromised 22.1 million people, federal authorities say”, Ellen Nakishama http://wapo.st/1VsS4Cf ))

What can we learn from this? There will always be hackers. But how seriously do you take security? Are you keeping personal data? Ensure your team is considering non-functional requirements and looking at security as well. Two major categories in security to look at are authorization and abuse. How is the system protected from typical attack methods like forceful browsing directly to the URL or manipulating values to circumvent your authorization controls? How could your system be abused by authorized (and, of course, unauthorized) users? (Example: massive file uploads to bog the system?) You need to discuss safeguards to prevent and detect the abuse or inappropriate use of your system. Also consider how things evolve. I bet when my background check was done (cough) years ago, the thought of China someone hacking into our government’s databases wasn’t even … Oh, I can’t even finish that thought. Just know it is incredibly expensive to have to react to and manage a disaster like this (to the tune of over $130 million).

#3 - Toys: The Hottest Item? Hoverboards (Pun Intended)

Image Source: http://bit.ly/1UrWd9o Image Source: http://bit.ly/1UrWd9o[/caption]

You’ve probably heard about this toy being pulled from Amazon and being banned, oh, everywhere. There seems to be a tiny problem with the lithium ion battery bursting into flames while charging.

What can we learn from this? While this may sound like a hardware issue, there are some software testing lessons here. Anything that takes multiple parts to make means you not only need to test the sum of the parts, but the parts themselves. In my world, I’m talking third-party apps or plugins that enhance our product. Do you build a text editor from scratch? Or (most likely), did you decide not to re-invent the hoverboard wheel and use a third party editor like TinyMCE? But I bet you tested that third party plugin (or at least ensured that they did a lot of testing themselves). Another takeaway from this — Do mobile stress tests on the battery! I’ll never forget one session when I was testing a feature on a mobile app, and by the end of the session, my phone was BURNING UP and my battery had almost completely drained (from a full charge) after just 20 minutes of using the app. (No, we did not release that version). Not only do you need to test your apps for functionality, but also how they impact your device!

#4 - Automotive: Takata’s Airbag Systems

Image Source: http://bit.ly/1PkjFVZ Image Source: http://bit.ly/1PkjFVZ

2015 was a bad year in automobile recalls, but probably none worse than the Takata Airbag System recall. In short, the Takata airbag inflators are causing the airbag to rupture, sending shrapnel into the vehicle. While the cause is being narrowed down, some factors include poor quality control at manufacturing time and other external factors like exposure to high heat and humidity.((Consumer Reports, “Everything you need to know about the Takata airbag recall”, http://bit.ly/1E8HOaC)) Takata has to replace airbag systems in over 30 million vehicles.

What can we learn from this? While this is not a software issue, this is an extreme example of a failure to implement quality control. Lack of quality control or a breakdown in understanding client environments for your software can be costly (in the millions). Consider that you are working on a complex system. Do you only use pristine data and environments? Or do you really consider its performance and stress your software? Continue those non-functional requirement discussions and think about stability, throughput, and response time (to name a few). Oh, and if you don’t see me out and about for awhile, it’s because I have to take in my car as it has been recalled to replace the airbag.

#5 - Online Gaming: Steam glitch shows access to other accounts

Image source: http://bit.ly/1SRJ4ZF Image source: http://bit.ly/1SRJ4ZF

As I take a look at various industries, let’s see what snafu caught our attention in the world of online gaming. Enter Steam, the online marketplace for PC games. The issue boils down to account users suddenly being able to access other people’s accounts, and in some cases, credit card information. Valve (the company that runs Steam) only indicates it was a configuration issue. It had an impact for about an hour (when they shut down the site completely), and they were back up in a couple of hours.

What can we learn from this? Configuration changes matter! I won’t repeat my thoughts on the importance of security testing here (as we talked about in the OPM data breach), but any tweak in configuration can totally change what your users see. It’s not enough to just test a login scenario. Test that you have access to the proper things and are not exposed to innappropriate information.

The year in review

What were your favorite quality disasters of the year? I look forward to what 2016 will bring! Ashley Hunsberger is a Quality Architect at Blackboard, Inc. and co-founder of Quality Element. She’s passionate about making an impact in education and loves coaching team members in product and client-focused quality practices.  Most recently, she has focused on test strategy implementation and training, development process efficiencies, and preaching Test Driven Development to anyone that will listen.  In her downtime, she loves to travel, read, quilt, hike, and spend time with her family.

Written by

Ashley Hunsberger

Topics

Software Testing