Speedo - Your Sauce Performance Sidekick

Posted by Christian Bromann in Performance TestingSauce Product Info

As part of the launch of Sauce Performance, a single solution for both front-end functional and front-end performance metrics, we developed Speedo. A tool that enables Sauce Performance to integrate with any CI/CD system.

In our research, we found one of the biggest obstacles to getting started with shifting performance testing left was the lack of proper tooling and guides for best practices on things like what to test, validate and optimize. Speedo fills this gap as a single tool to run and validate web app performance.

To get up and running with Speedo you can either use our Quay.io Docker image or install it via NPM (Node.js v8 or later required):

$ npm install speedo -g

This will install the package on your system and will make the speedo command available to you. The following examples assume that you have SAUCE_USERNAME and SAUCE_ACCESS_KEY exposed in your environment. If not ,you can pass them in as command line parameters -u <username> and -k <access key> or export them via:

$ export SAUCE_USERNAME=Slavocado
$ export SAUCE_ACCESS_KEY=XXXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX

The simplest way to make use of the Sauce Performance feature is to use the Speedo run command. It accepts the URL of the web page you want to check performance for as parameter and takes care of everything required along the way. Just run:

$ speedo run https://www.saucedemo.com

If you are running functional tests on Sauce as a CI/CD step before your performance test you can also attach the performance test to the build and give it a specific name such as:

$ speedo run https://www.saucedemo.com -b $BUILD_ID -n “Performance Test”

The run command takes care of everything that is important. It creates a baseline if you don’t have one for that specific page you are testing, downloads the performance logs for you as test artifact, and updates the job status. In addition, you can parameterize the run command to change the platform and Chrome version and also specify specific metrics you want to analyze. With the new Sauce Labs datacenter located in Europe (Frankfurt) you can even choose the location where your tests are executed by passing in the  --region parameter.

After the command is finished running, the tool prints out the results of the performance test. It highlights the metrics you have chosen to validate and prints out any difference to the baseline if a metric outlier is detected.  Depending on whether the test passes or fails, we exit the command with a proper exit code so that your pipeline can potentially block the release of your web application in case a performance regression was introduced.

Analyze Performance Data

Speedo can be used similarly to other performance testing tools such as  Lighthouse or WebPageTest; in addition, the analyze command will also allow you to examine the performance of any automated test that you run on Sauce Labs using the Chrome browser. With an automated performance test on Sauce you are able to bring your application into a specific state, get behind a login or just allow the browser to cache all assets in advance in order to reduce the network noise. Then using the Speedo analyze command you can assert the performance of any page load or page transition that happens during such automated test. It is important to point out that performance tests should be seen as an additional layer of quality assurance and mixing functional and performance tests together usually cause flaky results and unnecessary test coverage.

While common performance testing tools only allow validating the performance of one specific page URL, Sauce Performance with Speedo can give you access to any page that can be accessed with some automated steps using tools like Selenium or WebdriverIO. Given the following example of a Node.JS script that logs into an Instagram account:

import { remote } from 'webdriverio';

let browser

(async () => {
   browser = await remote({
       user: process.env.SAUCE_USERNAME,
       key: process.env.SAUCE_ACCESS_KEY,
       capabilities: {
           browserName: 'chrome',
           platformName: 'Windows 10',
           browserVersion: 'latest',
           extendedDebugging: true,
           capturePerformance: true,
           name: “Instagram Performance Test”
       }
   })

   await browser.url('https://www.instagram.com/accounts/login')

   const username = await browser.$('input[name="username"]')
   await username.setValue('performancetestaccount')

   const password = await browser.$('input[name="password"]')
   await password.setValue('testpass')

   const submitBtn = await browser.$('button[type="submit"]')
   await submitBtn.click()

   // dismiss notification alert
   const turnOnBtn = await browser.$('button=Turn On')
   await turnOnBtn.click()

   const profilIcon = await browser.$('span[aria-label="Profil"]')
   await profilIcon.click(

   await browser.deleteSession()
})().catch(async (e) => {
   console.error(e)
   await browser.deleteSession()
})

In the example script above we put in our login credentials, click on the submit button, and at the end, navigate to the profile page by clicking on the profile icon. In this scenario we would capture the performance of the following pages:

With Sauce Performance we not only capture page loads that have been initiated by the navigate command of your WebDriver (e.g. Selenium) script, we also capture page loads that got triggered by automation actions like clicks.

By default, the Speedo tool validates against page load times, however, Sauce Performance captures a wide variety of performance metrics (time to first meaningful paint, time to first interactive, etc.). Users can decide which metrics to validate as part of the performance check by including a complete list alongside the --metrics flag or to validate against all, simply call --all.   

$ speedo analyze "Instagram Performance Test" -u https://www.instagram.com/ --all

With this command, we are asserting all metrics that have been captured when we transitioned to the main page. It is also possible to pick a page based on the order it has been visited using the --orderIndex parameter.

Speedo Within CI/CD
Speedo has been developed in a way so that it organically fits into any continuous integration and delivery pipeline. By using our published Docker image, you can easily integrate it into Jenkins, Bamboo or CircleCI. The following example shows a regular Jenkins pipeline with linting, unit, functional and performance tests using the Speedo image:

pipeline {
   agent none
   stages {
       stage('Linting') {
           ...
       }
       stage('Unit Tests') {
           ...
       }
       stage('Functional Tests') {
           ...
       }
       stage('Performance Tests') {
           agent {
               docker { image 'quay.io/saucelabs/speedo' }
           }
           steps {
               sh 'speedo run https://www.saucedemo.com -u ${SAUCE_USERNAME} -k ${SAUCE_ACCESS_KEY} -b ${BUILD_NUMBER}'
           }
       }
   }
}

With that, you don’t need to install any dependency to run Speedo in CI/CD. Just specify the Docker image from Quay and you are good to go. Of course, this also works with a secure Sauce Connect Proxy tunnel to test local web apps or applications behind firewalls.

Speedo is one of many open source projects Sauce Labs develops and maintains. Should you have any issues, feel free to contact us via GitHub or Customer Support. We look forward to your feedback. We love Open Source!

Discuss: Speedo - Your Sauce Performance Sidekick
0 Comments

Free Trial

Get access to a free 14-day trial version, or contact Sales for more information.