7 Tips For Maximizing Your Sauce Experience

Here at Sauce, we're dedicated to helping teams bring quality applications to market faster and more cost-effectively. Our customers use Sauce in a variety of ways based on their development and deployment processes and setups, but there are a few practices that are common to a significant number of our users who run automated tests. We'll talk about what 7 of these best practices are and their benefits.

To do this, let's look at an exemplary Sauce account. AngularJS, the popular open source JavaScript framework, uses Sauce to run a variety of different types of tests. Because the Angular team uses Open Sauce, our free plan for open source projects, their tests are public, so we'll look at how they've set up their Sauce account to illustrate some good practices. You can see their public tests page here.

As you can see in the screenshot above, Angular is passing pass/fail statuses of their tests to Sauce, so their "Results" column shows passed, failed, and errored tests. Doing this allows you to quickly scan your tests to see what's failed and to investigate those failures. To record pass/fail status, set the passed flag on the job using one of our alternative job annotation methods, such as our REST API. If you're using one of our recommended frameworks or libraries for Selenium testing, as in Tip #6 below, pass/fail status could be passing automatically to Sauce. Read more about recording pass/fail status here.

Angular is using Travis for some of its builds, and has integrated it with Sauce to run tests with their build. Integrating your testing with CI allows you to catch defects earlier, making them easier to fix and saving you time and resources.  Sauce was built to enhance the value of a CI/CD workflow by giving you the scale and platform access you need, so we highly recommend integrating with your CI system to get the most value out of Sauce.  We have integrations for the most popular CI systems, including Jenkins, Travis, Bamboo, TeamCity, and CircleCI.

Once you have your Sauce integrated with your CI system, it's a good idea to pass build info to Sauce. You can see that Angular is doing just that, as they have build numbers showing in the Build column of their test page. Passing build info can make it easier to investigate failures and find the tests you're interested in. The most useful way to use the build field is to identify individual runs of a build with serial numbers or unique IDs, as opposed to branches or build rigs that will re-build many times, as this allows you to investigate specific runs. Build info will become even more useful as we roll our a series of updates to bring our UI in line with a familiar CI experience. We'll be starting with a completely redesigned dashboard designed to aggregate tests into builds, similar to what you see on your CI/CD dashboard. Passing build numbers will allow you to take full advantage of the benefits the updates will have to offer. Learn more about passing build numbers here.

By running your tests in parallel, you can take advantage of Sauce's large scale to speed up your test runs. You can see that the Angular team runs tests in parallel across a variety of platform configurations. Speeding up your test runs offers several benefits. First, you can speed up your testing, potentially reducing your testing cycle from days to minutes, so your developers have less idle time waiting for to see test results. Second, faster test runs means you can test against a wide variety of platform configurations simultaneously to ensure a great experience for every user. The method for running tests in parallel can differ depending on what language you're using, but you can find information about running tests for most languages in parallel in the tutorials section of the docs.

Security is a top concern at Sauce, and we take many measures to make sure your data is secure. One of these is Sauce Connect, our secure tunneling technology, which allows you to connect to staged apps behind a firewall while maintaining control of proxy and access policies. The screenshot above shows the test details page of one of Angular's tests, showing that they are using Sauce Connect. Even if you're not particularly concerned with security or you aren't testing behind a firewall, using Sauce Connect can be useful, as tests run through Sauce Connect may perform better. You can download Sauce Connect and find more information in the docs.

Of course, Sauce is compatible with all popular languages and frameworks, so you're free to use whatever works for you and your team. But, if you're trying to decide on a framework, we have recommended frameworks for each language that work particularly easily with Sauce. Many of them automatically provide convenient functionality like pass/fail reporting. These are the frameworks we recommend for different languages:

You can find more information about the recommended frameworks in the tutorials sections in the docs.

A useful way to investigate errors is to take control of a running session with the use of breakpoints. Breakpoints allow you to tell the browser to stop listening for Selenium or Appium commands, effectively pausing the test and allowing you to manually investigate a problem. You can use breakpoints to do things like investigate JavaScript errors or clear popups blocking your test.

To breakpoint a test, go to the test page of a running test and click the "Breakpoint Job" button. When you're done investigating the test, simply click the "Stop Job" button to shut down the VM. Jobs that have been breakpointed will be marked as such on your dashboard so you know they've been taken off Selenium autopilot during the run.

We hope that these tips help you maximize your Sauce testing. By setting up your account similar to Angular's setup, you'll be able to more easily maximize the benefit you get out of Sauce.