Cross-browser Testing (in your Browser!) with Sauce Scout

I blame John Resig for this. It all started last year at TXJS 2010, organized by Rebecca Murphey. I hosted a round-table discussion on testing with the leads of various JavaScript-based projects. Peter Higgins from Dojo, Ed Spencer from Sencha (formerly Ext JS), John Resig from jQuery, and me, representing Selenium.

When the topic came to manual vs automated testing, John made a surprising comment. “There will always be manual testing. Even if you could automate everything with robots, there are just some things only a human can do,” he said. He mentioned specific examples where humans excel, but robots (currently) do not. Qualitative things like watching to see if there is too much lag in an on-screen animation. It was “surprising” in that here was a person who (like me) cared enough about browser testing and automation to write his own automated testing tools (not once, but twice), and yet was strongly advocating the need for manual testing. His comments got me thinking if I’ve been too narrow-minded in my thoughts on software testing and if my own cloud testing startup is doing enough to help users solve all their testing problems. Automated testing is part of the solution, but there’s so much more we can do.

Fast forward to today. Sauce Labs is proud to announce Sauce Scout. Use it to drive any browser in your browser. Use it to check CSS rendering in any browser. Or do any kind of manual or exploratory cross-browser testing, leveraging all of Sauce Labs’ browsers-in-the-cloud infrastructure. A video gets captured  of every Scout session, and there’s a convenient “Take a snapshot” button to capture a screenshot when you find a bug and want to quickly flag it. After you end the session, you can embed the video or screenshots in your bug tracking system. Until today, the only way to use Sauce Labs’ OnDemand cloud of browsers was by controlling our machines via the Selenium API in any of the supported languages. While we continue to improve our API support in OnDemand (we rolled out Selenium 2 API support in February), we’re proud to open our service to more users than just those skilled in the Selenium arts.

Sauce Scout, at a low level, leverages the VNC protocol to allow users to directly drive our cloud machines via keyboard and mouse. We use an in-browser VNC client so there are no downloads required. And most importantly, no Selenium experience is required to start using Scout.

The heavy lifting to get all this to work actually happened last year when we released Sauce TV. With Sauce TV, we enabled read-only VNC access on all our cloud machines so you could watch Selenium automatically drive our browsers in the cloud.  At the time, we disabled manual control via keyboard and mouse — which we’ve now re-enabled for Scout. We also had to write our VNC proxy servers to create one-time VNC passwords for access, write code to kick out users after the session ended, and terminate (throw away) the virtual machine after each session ended. We do these things to improve security of the machines. Scout users have complete control over the browser during a session. Anything anyone does on the machine won’t affect other users.

If you do manual or exploratory testing and you no longer want to set up and maintain your own testing infrastructure, you should try Sauce Scout. Let us know what you think. If you do find any bugs, feel free to use Scout to create the bug report! You’ll need a Sauce Labs account.  Sign up for an account, if you don’t already have one. We offer accounts starting at $0/month.

Related posts:
Ridiculously easy cross-browser testing with Sauce IDE
News: Sauce Labs Streamlines Manual Software Testing in the Cloud With Sauce Scout
Sauce Labs Adds Sauce TV to Its Cloud-based Cross-Browser Testing Service
Sauce Labs Eliminates Barrier to Automated Cross-Browser Testing with Sauce Builder
Cucumber Sauce: Cross-browser testing in parallel