Want to extend your testing routine to include production? Canary release pipelines may be the solution you need.
This article explains what canary release pipelines mean and how they can help you test in production in order to achieve continuous testing.
What Is a Canary Release Pipeline?
Image credit: Greg Sypolt
A canary release pipeline is a deployment tactic to test in production to reduce the risk of introducing a new application issue in production. The intent of canary release pipelines and testing is to get an idea of how an application will perform in production by gradually routing a small percentage of users to the new version before making it available to the entire consumer base.
A deployment strategy that includes canary releases allows teams to find problems sooner rather than later by shifting testing to the right—in other words, into production. This doesn’t mean that you shouldn’t also test earlier in the development pipeline, of course. But by adding testing in production, you gain another quality guard around your application. The new tactic of testing in production is an essential piece of your testing strategy to deliver quality apps to customers continuously.
Image credit: Greg Sypolt
Canary Release Pipeline and Testing
With a canary release pipeline in place, you can perform canary testing.
Canary testing focuses on sending a small percentage of production traffic to the new version environment in production (to some number of users) and performing a pre-defined list of canary tests.
They typically include testing for the following:
Infrastructure
API
Performance
Lightweight User Acceptance
Visual
Application availability
This technique reduces the risk of introducing new issues with continuous testing methods and additional monitoring—by gradually routing a small subset of users to the new version of the application.
Image credit: Greg Sypolt
Post-Canary Production Testing
The testing train doesn't stop after canary releases. Once a canary release has occurred and you have completed canary tests, you should continue to perform testing, logging, and monitoring in production to expose and identify issues within the application, network, or infrastructure to help fix them before end users are affected. Here are a couple of testing methods you may consider for canary post-release deployment in a live production environment:
Teeing is similar to TAP compare where we send traffic to multiple servers. In this case, we send traffic to development servers for debugging.
Chaos Testing is where you want to introduce a failure to guarantee your application, network, or infrastructure can recover from intentional disaster. Yes—we want to break the application or infrastructure deliberately to ensure it automatically recovers (self-healing).
Conclusion
Traditionally, testing has been something that occurred to the “left” of application release. With canary release pipelines, however, you can test effectively in production while minimizing the risk associated with running tests on production applications. While you should absolutely continue testing pre-release, canary tests are a useful strategy for helping to achieve the goal of continuous testing, which provides the most robust, comprehensive test coverage.
Greg Sypolt (@gregsypolt) is Director of Quality Engineering at Gannett | USA Today Network, a Fixate IO Contributor, and co-founder of Quality Element. He is responsible for test automation solutions, test coverage (from unit to end-to-end), and continuous integration across all Gannett | USA Today Network products, and has helped change the testing approach from manual to automated testing across several products at Gannett | USA Today Network. To determine improvements and testing gaps, he conducted a face-to-face interview survey process to understand all product development and deployment processes, testing strategies, tooling, and interactive in-house training programs.
