QA's Impact on Application Security

appium group laptops

Who's responsible for application security?

In a world where DevSecOps is on its way to becoming standard practice, major elements of that responsibility are (or should be) obvious.  To the degree that security remains a distinct role, team members who occupy that role will continue to bear a large part of the responsibility.  Certainly, though, much of it also lies with the design team, along with the programmers, and just as certainly, considerable responsibility falls on the operation end of the delivery chain.  

What About Quality Assurance?  

In real-world DevSecOps, what impact does QA have on application security?  Is it simply a minor (and semi-passive) partner in the process, or can QA play a major role?  

As it turns out, Quality Assurance can and generally should have a significant impact on application security; QA in fact occupies a uniquely pivotal position in the process of developing and implementing security at the level of application and infrastructure code.

The Importance of QA

What makes QA so important when it comes to application security?

Consider what is perhaps the most distinctive set of requirements for almost any implementation of security in software form, whether it is part of application or infrastructure code: it must detect and counter malicious actions which may be far outside the range of the application's intended use cases, and which may target vulnerabilities in any part of the application, including those which do not appear to be closely associated with the sensitive information or control systems which are likely to be the actual goals of an attack.

Security Is Not Everyday Use

To put it another way, while most use cases focus on everyday actions by users with non-malicious intentions, application security use cases must cover out-of-the-ordinary actions by malicious actors.  When you add to this the fact that application security is by nature a mission-critical domain, it is easy to see that its adequacy and effectiveness on a functional level is not something that should be taken for granted.  If it fails to deliver as required, the cost may be very high.

QA's Basic Role

Now consider the broad categories under which most QA tests fall:

  • Code-level functionality - does it contain bugs, errors, etc.?

  • Requirements-level functionality - does it do everything that it's supposed to do?

  • Performance and stress - how does it affect the system, how does it perform, how does it handle low-resource situations, etc.?

  • User experience - how does it affect the user, either through direct interaction or indirectly?

In practice, most testing tends to focus on code-level functionality, performance, and stress, with requirements-level functionality and user experience often occupying much less time and effort; in general, this seemingly disproportionate allocation of time and effort works out fairly well.

What Security Requires

When it comes to applications security, however, this imbalance needs to shift toward requirements-level functionality, and to some degree, toward user experience.  Why?

With most software features, the process of testing whether or not an application meets its functional requirements is relatively simple in concept, if not in practice - you run it through a set of tasks representing reasonable use cases and compare actual outputs with expected results.

With security features, as we have seen, there is no reasonable use.  Instead, you must deal with misuse/abuse cases, which are often not reasonable at all in the everyday-user sense, and which may be very difficult to anticipate.  

Tests to determine whether security features meet their functional specifications must be based on an open-ended and growing list of out-of-the-ordinary, hard-to-anticipate, abnormal, and often malicious potential interactions with the software.  

Who's Responsible?

In organizations where there is a team dedicated specifically to implementing and maintaining security features, that team will typically be responsible for much of the testing conducted in connection with those features, particularly at that black-hat/challenge level.  

But by its nature, QA's domain includes exhaustive, thorough, and highly automated testing, so the most natural way to implement DevSecOps at the testing level is for the Security and QA teams to work in close coordination.

The Cost of Inadequate Security

This coordinated testing is crucial.  Along with your Security team's own test regime, QA is your first and most important line of defense against the consequences of failure to deliver the required security functionality.  

This failure in turn can place your organization at risk not just for financial and civil penalties, but depending on the field in which you operate (for example, one where public safety may be affected), possible criminal penalties as well.  Even without being brought into court, your organization can suffer a major, and perhaps irrecoverable, loss of reputation if a major security breach becomes widely publicized.

Bringing QA on Board

So how can you bring QA fully on board with Security?

QA/Security Coordination

Make the coordination between QA and Security formal and clear.  This includes making functional security compliance a major, high-priority QA responsibility, and also making key QA skills (such as large-scale test automation) available to the Security team.  It also includes some kind of basic plan for allocating various types and levels of functional-compliance security testing to the Security and QA teams so that the workload is balanced and nothing falls between the cracks.

Misuse/Abuse Cases

Adopt the misuse/abuse case model and work with Security to develop a full (or as full as possible) set of such cases, both for design and testing.  Whichever term one prefers (misuse case or abuse case), in practice they refer to essentially the same model: a use-case-like description of actions which can result in a breach of security.  Misuse/abuse cases should play a key part in the design of both security features and security-related tests.

Security-Conscious Testing

Incorporate security-related testing into all areas of QA.  Among other things, this means looking for potential back doors at both the level of code functionality (data type checking, input validation, overflow and buffer vulnerabilities, etc.) and performance/stress testing (DDoS attacks, malicious shutdowns of crucial control systems, etc.).  

Security and the User

A comprehensive approach to security in QA also means testing for the effects of security features on user experience, both to prevent loss of users as a result of unnecessarily clumsy implementation of those features and to look for ways in which users may try to circumvent those features.  Invisible security is often the best security, and good security testing at the QA level should include tests for that kind of invisibility.

QA is Key

What then is the impact of QA on application security?  In a DevSecOps world, it should be nothing less than major.  If your QA team is effectively sidelined when it comes to security, now is the time to bring it onboard, to bring it into full partnership with your security team, and to make it into the key player that it can and should be.

Michael Churchman started as a scriptwriter, editor, and producer during the anything-goes early years of the game industry.  He spent much of the 90s in the high-pressure bundled software industry, where the move from waterfall to faster release was well under way, and near-continuous release cycles and automated deployment were already de facto standards.  During that time he developed a semi-automated system for managing localization in over fifteen languages. For the past ten years, he has been involved in the analysis of software development processes and related engineering management issues.  He is a regular Fixate.io contributor.

Written by

Michael Churchman

Topics

Automated testingSoftware Testing

Categories