Massive changes in the development world are good and extreme for devs, but quality assurance (QA) teams are impacted as much, if not more. They may be taking on more tasks, looking at new tools, and thinking about new ways to execute their growing test suites. And looking forward, QA in the future looks much different than it does today. It is moving so fast that the changes - both good and bad - will be even more obvious by next year. Here is what QA looks like in 2016. The Bad At first a lot of the change will be disruptive in a way that is not all positive. The primary reason for this is because of a common confusion about what QA is, and what happens when it is not given the attention it deserves. This is where the top-down interest from decision makers is important.
Upper management will continue to trim teams. Talk of QA automation at the R&D management level has backfired a little. It has given the impression that automation is already in-place and ignores the strategy that should have driven it. Or, outsourcing and sticking with manual testing processes seems to be the easiest solution from some companies to scale. Both of these scenarios can lead to bad decision about the size and makeup of QA teams. What each case is missing is the detachment from the tests and the strategic objectives of the rest of the development team to move to more continuous processes. Outsourcing does not fit nicely into that automation without special effort that eventually will be put aside. And automation does not determine success without a team who understands the importance of testing and how to execute it to guide the process.
Application complexity will increase without additional QA efforts. Applications are getting more complex; not just in volume, but in the nature of the applications and the jumble of components they use. User flows are tied to even more comprehensive active pages. This means that functional testing is tasked with dealing with much more elaborate UIs. The backend relationship to customer-facing functionality is also compressed. Testing performed on these increasingly complex applications is expected to also meet the decreased time between releases. The only way to respond to this complexity is with a more comprehensive testing strategy, better tools, automation, and cloud-based testing infrastructure.
More bugs, more bugs! If I could graph the ratio of bugs to the number of application releases, I would find that overall the trend is a greater percentage of bugs per-release. This not what we want! For many organizations, the drive to move faster has had the unfortunate counter effect of decreasing quality. Even though bugs are reduced in severity, fixing them still requires effort, where the effort cost is drastically increased if the bug makes it to production. So if you look at a year to year comparison of efficiency, there is a direct correlation of the number of bugs and the number of releases per year. Not to mention team morale.
The Good While there are some difficult, negative pain points happening in this ecosystem, there are also a lot of positive things happening in the automation world. Teams should be especially excited about the movement from QA to QE, a more strategy-focused initiative. These shifts are going to benefit everyone.
QA is everyone's job, and QE is the facilitator. Everyone on the team is a part of quality assurance, and in 2016 they will realize it. When apps are released with bugs, developers get much for heat for them, faster. This is because customers have access to public channels that enable them to complain about bugs (app stores, reviews, social media). Mobile is in part driving expectations of the market, so that customers expect instant gratification and high quality apps. Lastly, applications are much closer to companies’ line of business, thus the bottom-line. IT will also feel the pressure, and are often the first line of defence. But they can’t help without better communication with Developers, which means a greater shared effort to try to isolate problems faster - not more barriers. QA will start to make the transition to QE (Quality Engineering), which puts the focus on testing strategy and architecture over doing specific testing. And QE has the rare point of view of seeing the entire flow of code. This puts them in a great position to provide strategy versus just testing, to facilitate that IT to Dev communication, and to help devs identify bugs BEFORE they are released.
Applications start with quality. Why does QA have to be at the end of the pipeline? With the new Continuous Integration (CI) process in place, QA efforts move upstream a little, but ideally even more so. QA will help Front-End teams to run quick functional tests in-place, and because it is possible, have their own functional testing sandbox to run their compiled branch of code against the full stack before it is sent to a shared CI environment. This small change catches bugs faster on two levels. First is because the code is hot in the developer’s mind, and they already have a visual of how it should work. Second is that local bugs are much easier to test when run separately from the entire stack. Then the bugs found in broader end-to-end testing are easier to resolve, which is a much better time spend. Finally, just the opportunity to catch bugs faster helps prevent them from reaching production.
The role of QE expands to team-wide leadership. In development, having the whole picture of the entire stack and deep a understanding of the product is rare. Each developer knows their piece, but does not often see it in action from the consumer point of view, if at all. QA teams have the whole picture, even if they do not want to. They see the product so often and are using Selenium to reverse into it that they know it better than anyone. They know all the infrastructure, components, and the relationship to the backlog and roadmap. Taking this holistic point of view and using it as a feedback loop to the broader team helps spearhead problems like a performance drop or enables them to offer suggestions. I believe QA as a strategy will be more prevalent over just testing.
The Enterprise A lot of that sounds good. But what does it look like in practice in an enterprise?
IT might have the budget but may not be the decision maker. As QA/QE teams strategically influence the direction of the product, instead of being situated just before release, they gain purchasing power. They will be bigger technical influencers with regard to the tools used and how they are implemented. Although often the budget for these tools is controlled by IT, ultimately the vetting and recommendation is taken from the QA leadership without much question.
Management cares. Management will feel the pain of poor software quality. Hopefully they learn the easy way, but often that is not true. I alone have been guilty of a sharing various application complaints over social media. And now organizations have social media teams that respond quickly, thus acknowledging my issue. I would like to think my complaints are issued in more detail than the typical user’s application rage. But I am not in a minority. Users are critical, and even small issues cause serious frustration. The efficiency of sharing these issues is high. This is just the new norm, and to management it is more visible and embarrassing. The only outcome is to take quality more seriously.
The benefits of automation will be clear. Questions about how to implement automation into an existing organization is usually the largest impediment that Enterprises face. But with the above demands, clearly automation has to take some portion of QA efforts. Everyone involved will want to understand what tools are available, how they work, and why they are not already being used. Hopefully an “I told you so” moment for QA. Yes, manual testing will be impacted - but likely in a positive way, where they can focus on exploratory testing against a roadmap and backlog, versus monotonous testing of age old common functionality that is better served through automation.
In my opinion front-end developers and QA/QE teams need to spearhead the change, or at least be prepared for it. One of the easiest ways to do that is a pilot that has a clear beginning and end, and shows the value of strategy and automation. The pilot can automated the test of one or two major components of the application, most likely standard and un-changed functionality. In the test you will need to arbitrarily create a bug. And measure the effort cost of testing with the bug, and without, compared to a manual test for the same issue. It should not be difficult to show that through automation, more bugs will be caught in less time. In order to convince a broader team, results and metrics are more important than how it is done. The other, rather fun challenge is mobile. Mobile versions of existing web applications, or brand new mobile applications that must fit into the existing test matrix, requiring yet another big change. Mobile requires new processes but with the exact same goals, and very often the same suite of test, just with a twist. There is no slowing down modern development’s growth and impact on teams. All an organization can do is keep an eye on the ball, get out of the weeds a little, and facilitate and manage change. This is going to be an iterative process, but in 2016 it’s no longer speculation; the expectations of releasing more at the same or higher quality is going to be required. By Chris Riley Chris Riley is a technologist who has spent 12 years helping organizations transition from traditional development practices to a modern set of culture, processes and tooling. In addition to being a research analyst, he is an O’Reilly author, regular speaker, and subject matter expert in the areas of DevOps Strategy and culture and Enterprise Content Management. Chris believes the biggest challenges faced in the tech market is not tools, but rather people and planning. Throughout Chris’s career he has crossed the roles of marketing, product management, and engineering to gain a unique perspective of how the deeply technical is used to solve real-world problems. By working with both early adopters and late, he has watched technologies mature from rough solutions to essential and transparent. In addition to spending his time understanding the market he helps ISVs selling B2D and practitioner of DevOps Strategy. He is interested in machine-learning, and the intersection of BigData and Information Management.
