Unit Testing Chef Cookbooks with ChefSpec

I recently had the opportunity to get involved with Chef cookbook development, and begin learning how to write some tests for infrastructure code by using various testing frameworks. One critical aspect of developing any code is writing unit tests. We use ChefSpec - it's a unit testing framework for testing Chef cookbooks.

ChefSpec is simple to write. It reads a whole lot easier than other unit testing frameworks, and provides fast feedback on cookbook changes without the need for a virtual machine or cloud machine.

Below, I'll show off ChefSpec in action. I'll do that by first setting up a workstation to run Chef locally, then use the existing Chef community cookbook to deploy the application on the local workstation environment. Finally, I'll walk through some of the unit to make sure everything is working properly.

To set up your local development environment, you'll first need to download the Chef Development Kit (DK). Why? I can specifically download Chef DK version 0.15.15 since I know it is compatible with my environment. The version will change as the Chef DK receives new updates (and it's your responsibility to validate the changes are compatible). Once you've downloaded and installed Chef DK, open up your terminal window and check the Chef version installed.

$ chef -v Chef Development Kit Version: 0.15.15 chef-client version: 12.11.18 delivery version: 0.0.23 berks version: 4.3.5 kitchen version: 1.10.0

Once you've downloaded and extracted the Chef Selenium community cookbook from the Chef Supermarket, open up your terminal window and navigate to the chef-selenium directory to finish preparing your local environment.

$ cd ~/work/chef-selenium $ chef exec bundle install ... ... Bundle complete! 10 Gemfile dependencies, 113 gems now installed. Use bundle show [gemname] to see where a bundled gem is installed.

ChefSpec allows us to simulate the convergence of resources to prove that the cookbook code works correctly before pushing changes to production. It's much easier and cheaper to catch bugs locally, as opposed to having a production application crash and burn. ChefSpec is the fastest way to test resources and recipes as part of a simulated chef-client on a local developer machine. It is an extension of RSpec, a behavior-driven development (BDD) framework that allows writing readable Ruby tests. Teams should aim for 100% coverage of code for unit testing. Every Chef cookbook should have a folder called “spec.” (This is where ChefSpec unit tests live). Let's review the chef-selenium file structure for unit testing:

|- chef-selenium/ |-- recipes/ |---- default.rb |---- hub.rb |---- node.rb |-- ... |-- spec/ |---- unit/ |------ default_spec.rb |------ hub_spec.rb |------ node_spec.rb |---- spec_helper.rb

When writing unit tests for your cookbook, the standard is to create a separate spec test for each recipe in the /spec folder.

We're only reviewing the spec test 'default_spec.rb' that describes the 'default.rb' recipe. This recipe downloads and installs the Selenium Standalone JAR file for Windows, Linux, and Mac OS. The SoloRunner allows us to simulate testing against the different operating systems (platforms) and versions by passing a parameter. Lines 6, 46, and 87 simulate the Chef run-in memory, as previously described, and override any defined node attributes that we'll use in our spec tests later.

6 ChefSpec::SoloRunner.new(platform: 'windows', version: '2008R2' do |node| ... ... 46 ChefSpec::SoloRunner.new(platform: 'centos', version: '7.0' do |node| ... 87 ChefSpec::SoloRunner.new(platform: 'mac_os_x', version: '10.10') do |node|

source: https://github.com/dhoer/chef-selenium/spec/unit/default_spec.rb (Lines 6, 46, and 87)

Let's look at the different ways to test whether a package resource is installed or not installed with ChefSpec. Line 1 tells Chef to unzip the package unless the platform is Windows or Mac OS.

1 package 'unzip' unless platform?('windows', 'mac_os_x')

source: https://github.com/dhoer/chef-selenium/recipes/default.rb (Line 1)

For the next test, Line 6 tells Chef to simulate a test run against Windows 2008R2 by calling the default recipe to determine if the package will be unzipped. In Line 14, by using ChefSpec PackageMatcher, you can validate that a package was not installed for the Windows platform.

Windows

6 ChefSpec::SoloRunner.new(platform: 'windows', version: '2008R2' do |node| ... 13 it 'does not install zip package' do 14 expect(chef_run).to_not install_package('unzip') 15 end

source: https://github.com/dhoer/chef-selenium/spec/unit/default_spec.rb (Line 6 and 13-15)

Next test - Line 57 proves that a package was installed for the Linux platform.

Linux

46 ChefSpec::SoloRunner.new(platform: 'centos', version: '7.0' do |node| ... 56 it 'installs package' do 57 expect(chef_run).to install_package('unzip') 58 end

source: https://github.com/dhoer/chef-selenium/spec/unit/default_spec.rb (Line 46 and 56-58)

Next test - Line 7 sets environment variable "SYSTEMDRIVE" for Windows, and Lines 8-9 “override” a node attribute [‘selenium’] [‘url’] that we’ll use in our specs later. Line 34 simulates the download of the Selenium Standalone JAR file by using the ChefSpec RemoteFileMatcher to verify the remote file was created.

7 ENV['SYSTEMDRIVE'] = 'C:' 8 node.override['selenium']['url'] = 9 'https://selenium-release.storage.googleapis.com/2.48/selenium-server-standalone-2.48.0.jar' ... ... 33 it 'downloads jar' do 34 expect(chef_run).to create_remote_file('C:/selenium/server/selenium-server-standalone-2.45.0.jar') 35 end

source: https://github.com/dhoer/chef-selenium/spec/unit/default_spec.rb (Line 7-9 and 33-35)

To run ChefSpec, from the terminal, go to the root of your cookbook (chef-selenium) and run Rake. As you can see by reviewing the console output, the chef-selenium cookbook does not have 100% test coverage, but all ChefSpec tests passed.

Terminal Console Output

$ chef exec bundle exec rake Finished in 23.33 seconds (files took 2.27 seconds to load) 52 examples, 0 failures ChefSpec Coverage report generated... Total Resources: 36 Touched Resources: 35 Touch Coverage: 97.22% Untouched Resources: remote_file[/opt/selenium/server/selenium-server-standalone-3.0.0.jar] selenium/recipes/default.rb:16

The test coverage output is disabled as the default setting. The following code snippet needs to be inside the spec/spec_helper.rb file to enable test coverage. I recommend enabling test coverage for any application code.

Enable ChefSpec Test Coverage

ChefSpec::Coverage.start!

source: https://github.com/dhoer/chef-selenium/spec/spec_helper.rb (Line 5)

In this blog post, we set up a clean development environment and reviewed some of the ChefSpec tests for the chef-selenium default recipe, executed them across multiple platforms, and analyzed the test coverage output. Your infrastructure code deserves testing, and before deploying those servers, prove that your cookbook works.

This blog post only scratches the surface of testing Chef cookbooks. Just taking the time to learn something new can be a daunting task. There are a lot of new concepts to learn and best practices to understand when developing and testing infrastructure code. To continue learning about ChefSpec and RSpec testing extensions, you can reference these documentation links to test cookbooks, recipes, and resources:

Greg Sypolt (@gregsypolt) is a senior engineer at Gannett and co-founder of Quality Element. He is a passionate automation engineer seeking to optimize software development quality, while coaching team members on how to write great automation scripts and helping the testing community become better testers. Greg has spent most of his career working on software quality — concentrating on web browsers, APIs, and mobile. For the past five years, he has focused on the creation and deployment of automated test strategies, frameworks, tools and platforms.