Our friends at The Able Few have been working on an exciting project, and they wanted to share how they’ve been using Sauce to test the product they are developing in partnership with Click with Me Now. Read on to hear more about how they integrated Sauce into their development process, and the open source GruntJS tool they built to help them use Sauce!
For the some time now, we have been developing an application over at The Able Few, a St. Louis & Indianapolis based product and software development company, called Click With Me Now. CWMN, is a no-download, co-browsing solution that allows users to share a browsing session with others in a single click.
In the early stages of development, we built a series of prototypes to serve as a proof-of-concept for the application when demoed in a controlled environment. We focused most of our initial efforts around Chrome/Webkit, obviously, which allowed us to cover an impressive amount of ground in a short amount of time. When it came time to start the full build of the application, however, we had to start backfilling support for other older browsers and make sure that this didn’t impair our existing codebase or slow us to a complete halt with testing.
After weeks devoted to dealing with countless browser compatibility issues, mostly in IE, and many profanity laced insults hurled at the computer gods, we had a working prototype that functioned in at least the latest version of every browser. Of course that wasn’t good enough and going forward we were going to need to be able to test the app in every other browser known to man.
We started writing out some Selenium and Capybara tests. This allowed us to do things like disable Websockets and Flash, in order to make sure that the application didn’t crash and burn, which we could test in Chrome, Firefox, and Safari without a hitch — but not IE! Also, what about mobile? Oh, and what would happen if the Host was using an old version of Firefox and the Guest was using a Webkit nightly? As these questions began to pile up, our aspirations of adequately testing our application began to sour. We played around with VirtualBox VMs, but it quickly became apparent that the number of OS and browser variants we needed would become a nightmare to manage, not to mention the licensing costs. We also needed to think about mobile devices, older versions of OSX, Linux, and myriad other combinations that we had yet to consider.
It was a lot to deal with and we certainly felt the pressure of needing to accomplish this in a timely manner. Then by a stroke of luck we came across Sauce Labs.
The Guest, in general, has a lower set of requirements. The relationship between the host and one or more guests make testing this application a real chore with significantly more points of failure than a typical web application.
Since we already had a good amount of test coverage in Selenium using Ruby, the effort required to integrate our existing tests with Sauce Labs was pretty minimal. Obviously, it defeats the purpose if all the testing isn’t automated, so we made it possible to pass arguments to our tests and instead of just running `ruby spec/test.rb` we are running `ruby spec/test.rb windows8firefox17`, and so on.
To further automate this, we created a JSON file that contained all of the browser/OS combinations that we wanted to test, along with the data that the Sauce Labs API needed. Of course, a couple of weeks later, we found out that Sauce Labs already has an up-to-date list available — one day we may update our code to use their version, but for now, our method does what we need it to do.
Now, we needed to parse the JSON into actual “identities” and we needed Ruby to understand those arguments. Being that I’m a JavaScript developer with a long history of PHP, let’s just say that my Ruby isn’t the best, but I needed to solve a problem. So, with the test code in hand, I took a stab at a solution. We ended up with something that actually worked better than I could have expected.
By updating `BrowserList.json` to point to your local JSON file from above you can test this code out, too — well, as long as your environment is setup and ready to run Sauce Labs tests with Selenium and Capybara. Just a couple updates to your Gemfile and a quick read through Sauce Labs’ ‘Getting Started’ guide in their Documents should get you started
Still, with all of this done, we were having to go through each and every browser by hand and run the Ruby spec tests, which you can imagine was pretty time consuming. This is where JavaScript came to the rescue! Thanks to Bocoup’s GruntJS, running automated tasks is now a no-brainer. Simply build a Gruntfile, give it a few lines of code, run a command, and lean back in your chair as your computer actually does the hard work. Or, in our case, dedicate a machine to running these tests because it’s going to take a LONG time.
Currently, there are no official methods of using Grunt with Sauce Labs so we decided to build our own solution and open source it. Qettlhup, Klingon for “sauce”, is an automated tool built on GruntJS that facilitates the testing of browsers in Sauce Labs using a JSON object.
Once you’re familiar with GruntJS — which you should be, because it is awesome — you will quickly see how it works. You simply pass it the language of the tests that you want to run, the path to the file that contains the tests, and the JSON file that lists all of the browsers that you want to run the tests against. In the case of CWMN we have two sets of tests, one for the Host and the other for the Guest, which both have their own requirements, so we pass Qettlhup another task in the same object and we’re ready to go. You can see examples of setting up single and multiple tasks on the Qettlhup README.
Now, we just type a single command — `grunt qettlhup` — and away it goes, running all of our tests for us and giving feedback on each set of tests as they run. The process will stop as soon as it hits an error, which gives us the opportunity to fix the problem and to test that fix against a specific browser outside of Qettlhup. Once the error has been resolved, we can go back and run our battery of tests again.
All of that said, even with our huge library of tests, custom GruntJS automation tool, and knowledge of Selenium and Capybara, none of this would have been possible without Sauce Labs. It is a utility that has saved us countless hours of manual processing and testing and has provided us with fantastic reporting and feedback. Each test even includes archives of image and video references. There is no telling where we would be if Sauce Labs wasn’t around! We couldn’t recommend their testing platform more and are looking at it with every build we do in the future.
