Security and Testing

Is your test environment secure? Do you know who has access to your test data, your source code, your design specifications? There was a time, back in the days of stand-alone test systems and networks that were strictly local-area, when those questions would have been easy to answer. A co-worker or two might have been looking over your shoulder, but that would have been about it. These days, however, applications are exposed to the public web, and such questions can have serious implications for your software's security and your company's bottom line. Software and IT companies may still have physical locations, but much of the development and testing is done off-site, by employees, contractors, and services that transfer data over the Internet, such as cloud-based testing solutions. That is why picking one who cares about your application security is important. Lets look at the risks, then at what a good solution looks like.

Most companies are reasonably security-conscious when it comes to software development, but does that security consciousness extend to your testing process and suite? For many companies, using an Internet-based testing service just makes good sense from both a practical and economic point of view. But when your QA department shifts its testing from in-house to online, do QA management and staff also shift their understanding of security from "stand-alone machines safe behind four walls" to "somewhere out there on the Internet"?

Consider what can be at stake if outside parties gain access to your test data or your test environment:

Unfortunately, not everybody in business sticks to the rules of fair play—and that includes the technology sector. Intellectual property is a valuable commodity, and for many technology providers, it is their main asset. This is particularly true for smaller companies and startups, which may not have had time to develop other assets, such as name-recognition or a reputation for high-quality products or services. IP is not hard to steal, since it generally consists of information that can be easily copied, compressed, encrypted, and transported. Often enough, sufficiently detailed (or even general) knowledge of an idea is sufficient for it to be stolen. If you are testing a feature or technology that is worth stealing, and if there is a way for IP thieves to gain access to your test environment, there is a reasonably good chance that they will break in. When that happens, you may find yourself not only competing with knockoffs of your software-in-progress, but also fighting in court to regain control of your proprietary technology and data.

If you're doing anything worthwhile or potentially money-making, you have competition—and if your competitors are smart (which they almost certainly are), they're watching you. They may play a cleaner game than IP thieves, but they want to know what you're planning, what new technologies and services you're in the process of implementing, how far along you are in developing them, and how well they're working. Needless to say, if they know what mistakes you're making, they can learn from them as quickly as you can. If your online test environment is not secure, it's possible that your competitors have access to your test results, and may even be able to observe your tests while they are in progress. A test suite can give a sense of code, functionality, and potential issues and weak spots in your application. They'll know what new features and technologies will be in your next release, and they'll have a head-start on adding competing versions to their own products. If they can learn enough from watching you, they may even beat you to the market.

Maybe you're not testing anything that can be easily cloned by competitors or stolen. Maybe you're working on things such as better implementations of known technology, and a new user interface that incorporates your brand-new, still-under-wraps company logo. There still may be some major security issues to keep in mind when it comes to online testing. Consider this all-too-familiar scenario: Hackers break into a site, dig out some attention-getting information, and spread it all over the Internet. Very often, it turns out that what was mildly entertaining to the hackers is embarrassing or even damaging to the people and organizations affected by the leak. Do you want your upcoming feature list to become public knowledge while those features are still in the early development phase? Do you want your raw test data posted for anybody to see? Do you want your pre-release bugs to become just another occasion for lulz? Just one little break-in, into a non-secure test environment, could do all of that.

How do you find a secure online test environment? What are the key elements of online test security? Besides such basics as a secure test infrastructure and physical on-site security, a truly secure online test service (such as Sauce Labs) should provide a suite of strong test-oriented security features:

If an online service does not offer security features such as these, you cannot count on it to provide you with a secure testing environment. A genuinely secure software testing service, on the other hand, can guarantee that all of your test data will truly be for your eyes only.

Michael Churchman started as a scriptwriter, editor, and producer during the anything-goes early years of the game industry. He spent much of the ‘90s in the high-pressure bundled software industry, where the move from waterfall to faster release was well under way, and near-continuous release cycles and automated deployment were already de facto standards. During that time he developed a semi-automated system for managing localization in over fifteen languages. For the past ten years, he has been involved in the analysis of software development processes and related engineering management issues.