Great news! The organization that you work for has finally seen the light and wants to start introducing automated functional testing across the entire enterprise. The unit tests that you have are a good start, but they are not enough to ensure working applications. The problem is that every area of the company has its own idea of what functional tests are and how they should be structured and executed. To make matters worse, there are hundreds of tools available, meaning that the enterprise roll out could go sour pretty quickly.
Your company could go with a pure shared services model where everything flows through a central point, but everyone can see that this could potentially lead to a bottleneck, and it will not be flexible enough to support all of the development models in use across the application portfolio.
After talking to a few consulting firms and other organizations which have already started the journey, your company decides that it should have a Center of Excellence (CoE) define the standards, pick the tools, and cross train anyone in the organization who wants to introduce automated testing to their build and deployment pipelines. Ideally, this will be added into an existing DevOps CoE which already has most of the tooling required to support the automation in place and has established relationships throughout the organization.
Functional testing is the glue between Dev and Ops
At a quick glance, it is true that QA needs its own CoE, since translating user stories and use cases into testable scenarios is a world unto itself. Yet, the tooling required to execute those tests also belongs with the rest of the development to operations pipeline.
QA is the cornerstone upon which any good DevOps practice is built. The glue that holds Dev and Ops together is not just the ability to build an application, but also the presence of enough functional tests to allow operations teams to relinquish some of their obsessive need to control everything going into production and allow continuous delivery and deployment to bring automation all the way to production. A full CI/CD pipeline needs functional testing, or else no one will trust what it is producing, and Dev and Ops will go back to playing the blame game.
The most successful DevOps Center of Excellence will be able to provide an end-to-end solution for taking developer code and having it run in production with no manual steps in between except the occasional approvals needed in some regulated environments.
Quality is the key to excellence
In its simplest form, a CI/CD pipeline executed through a DevOps practice starts with a code commit and ends with a built artifact that is running in an environment and ready for use. Even that environment can be (and often is) defined in code and configuration files so that it is reproduced every time.
Figure 1: Sample flow with no automation around functional testing
The above pipeline (figure 1) lacks automated testing for assessing the functionality of the application. In this scenario, we can be sure that an artifact was built. Thanks to unit tests, we can be sure that every individual component within the application works as specified. We can even be sure that the environment was successfully built with the application artifact running in it. However, we have no idea if the components work together or if they do what the application was designed to do. For all we know, it might be returning images of cats for every request. While cat lovers might see this as a great public service, if your app was built for a puppy adoptions agency, then it is clearly not working as intended. In this case, your app should not go live and you should kick off an automated rollback.
Figure 2: Sample flow with automated functional testing
Where to start
To introduce functional testing into your DevOps Center of Excellence, you should work with application development teams to find the most commonly used languages and pick the best combination of functional testing products to integrate into your standard toolset. There are lots of options available, and open source products are a good place to start. Some of the leading projects are Appium for mobile, Selenium for UI, and Soap UI for API testing.
Vince Power is a Solution Architect who has a focus on cloud adoption and technology implementations using open source-based technologies. He has extensive experience with core computing and networking (IaaS), identity and access management (IAM), application platforms (PaaS), and continuous delivery.