Posts Tagged ‘Continuous Integration’

Introducing Bamboo Sauce

November 8th, 2010 by Ashley Wilson

The Sauce Labs ecosystem continues to happily grow with the addition of a new CI plugin built specifically for Sauce OnDemand.

Bamboo Sauce, created by Systems Bliss engineer Jonathan Doklovic, enables Bamboo users to easily integrate with Sauce OnDemand to run builds across multiple browsers in parallel. Sign up for the live demo this Thursday at 11 am PST to see how this pairing will speed up your test cycles and reduce the need for costly test infrastructure, while leveraging your existing investment in Bamboo, the popular Atlassian CI server.

Setup is simple, needing only a one-time admin configuration, and builds can be customized to test on a variety of OS/Browser combinations. Bamboo Sauce takes care of Firewall problems by creating a direct SSH tunnel from the cloud to a specific server (when you use Sauce OnDemand, we never see your code – rather, it stays behind whatever guard you have in place), and it also lets you pick which tests you want recorded via Sauce TV.

To find out more, tune in to the live demo, led by Systems Bliss and Sauce Labs. Jonathan of Sysbliss will walk you through setting up Bamboo Sauce, and also answer any questions. To follow along step-by-step, download the plugin and sign up for a free 30 day trial of Sauce OnDemand.

(To note – Bamboo Sauce is still in beta. We’d love to hear any feedback you have, so please let us know!)

Share

Doing Continuous Integration Testing? Check Out Our CI API

November 4th, 2010 by Santiago Suarez Ordoñez

Testing is a key ingredient in CI. And so in the constant effort to make our service insanely flexible and easy to use, we’ve released a brand new API that will help Continuous Integration servers such as Hudson, Bamboo, and TeamCity send Sauce OnDemand some interesting information about each test they run.

This includes the pass/fail final status of the test, the build number in which it was run, and even your own custom data.

Since some of this new job info is known only when the test finishes (e.g. whether the job passed or not), we’ve created not only one but two alternative methods for the info to be sent – the REST API and Selenium’s standard setContext command. Both are documented in our docs, and anyone interested can use them right now.

Once the info makes it to our end, it will be appropriately displayed with each job, helping our users find and group their jobs with less effort. (Notice that some of the work needed to display this in a cool way is still in our TODO – but the data is there). You can see some example jobs [1] with this data set or just create your own!

In the near future, we’ll be working with the authors of the Hudson and Bamboo plugins to add support for these CI APIs.

If you’re interested in hearing more, trying early versions of these plugins, or would like to see a plugin for another CI system, feel free to ping us here!

[1] https://saucelabs.com/jobs/4baf3ad0cd8b686ef554135c4addf621,
https://saucelabs.com/jobs/0d925d14c23888b8561215360b70a78d

Share

Kohsuke Kawaguchi Demos Cross Browser Testing with Hudson + Sauce OnDemand

September 3rd, 2010 by Ashley Wilson

As part of our featured webinar series, Kohsuke Kawaguchi, creator of Hudson and founder of InfraDNA, recently led an interactive demonstration on using Hudson with Selenium and Sauce OnDemand. In case you missed the live presentation on September 1, check out the recorded video to learn more about:

1. Setting up the Hudson plugin for Sauce OnDemand
2. How using Hudson for CI will drastically speed up your testing process
3. Tips for integrating Hudson with Selenium Grid
4. Tips for integrating Hudson with Sauce OnDemand

Share

#SFSE Meetup Videos: Hudson with Selenium + Sauce OnDemand

July 20th, 2010 by Ashley Wilson

At our last #SFSE meetup at StumbleUpon, Kohsuke Kawaguchi, the creator of Hudson, gave a fantastic presentation on Hudson and the integrations with Selenium. He also unveiled a new Hudson plugin for Sauce OnDemand, documented here in an earlier blog post.

Hudson is an easy-to-use, open-source Continuous Integration (CI) server that supports distributed builds and multiple extensions, including those for Selenium. The first video below gives background on Hudson, the reasons for its development, and the features included. The second two videos walk you through using the Hudson Sauce OnDemand plugin, which makes the Hudson & Selenium integrations quicker and easier to utilize.

If you’d like to see this presentation in its entirety, check out our Sauce Labs YouTube channel. Thanks again to our friends at StumbleUpon for providing a great space for us to use for the event. To hear these talks in person, join our Selenium meetup group. We generally host meetups once a month around the Bay Area, featuring different Selenium topics and speakers, so join in the conversation!

Hudson Background:

Sauce OnDemand Plugin:

Share

Bromine + Sauce Labs: One Big Happy Integration

July 13th, 2010 by Ashley Wilson

The good people behind Bromine, an open-source QA tool that allows you to easily run and view your Selenium RC tests, recently created a Sauce Labs integration. This means you can now run your Bromine tests with Sauce Labs’ in-the-cloud service, Sauce OnDemand, with the added bonus of having a jobs queue AND video playback. Exciting news for all!

Check out the demo below, and download Bromine here to start using it with Sauce OnDemand today.

Share

Kohsuke Kawaguchi Announces New Hudson PlugIn for Sauce OnDemand

July 6th, 2010 by Ashley Wilson

By Kohsuke Kawaguchi

There has always been some overlap between Hudson users and Selenium users. As a result, the support for Selenium in Hudson, such as the Selenium Grid plugin, has generally been good. But while this Selenium Grid support, for example, does make it easier for you to run cross-platform, cross-browser tests, setting up the right OS/browser combinations, especially in the environment that doesn’t use virtualization, can be tedious.

This is where Sauce OnDemand comes into play. Sauce OnDemand solves this problem by offering browsers running in the cloud. But these browsers need to be able to talk to the web server being tested, which is most likely inside a firewall. Although Sauce OnDemand offers a SSH port forwarding option to handle this, setting up and and then tearing them down (or keeping them going all the time) is time consuming, especially when you are running your tests on a CI server that spans across multiple computers.

I wrote the new “Sauce OnDemand” plugin to fix this problem. With this plugin, Hudson will set up and tear down tunnels automatically for you.

Usage

Once you install a plugin from Hudson’s update center, go to the system configuration and specify your Sauce OnDemand credential:

Then, for each job that uses Sauce OnDemand, tell Hudson to set up SSH tunnels before the build. Hudson will tear them down at the end of the build.

Your projects don’t need anything in their build scripts to do this, which saves a lot of time if you have multiple projects/modules that use Sauce OnDemand. This also avoids the complexity of maintaining a persistent connection, which has its own complications. SSH tunnels on Sauce OnDemand starts very quickly, normally about 10 seconds, so the impact of this to the total build time is minimal.

Host name auto-allocation

Once you start using Sauce OnDemand extensively in Hudson, you’ll notice one problem. Normally you hard-code the host name that browsers connect to (which points to the cloud side of the tunnel and then forwards it back into your intranet to your webapp), but if you do that, you won’t be able to run such tests concurrently on Hudson, since the tunnel setting is global to your account.

In other words, imagine hard-coding your test to send HTTP requests to “example.org”, and you have two active branches of that project in development now. If the builds of those branches happen to run at the same time, their tunnel setup collides!

In the Hudson plugin, I added a mechanism to let Hudson auto-generate a unique host name when it sets up a tunnel. To do this, specify “AUTO” where you normally specify the domain name (“example.org” in the above example). Hudson will generate a random host name and make that host name available as the environment variable named SAUCE_ONDEMAND_HOST.

Your test should then look up this value and pass it to Selenium when you connect to it.

Under the hood

The implementation of this is done in two parts. First I wrote a library that performs SSH tunnel setup and tear down, and then I wrote a Hudson plug-in around it. The former is reusable on its own, and would be useful if you want to take the same idea and apply it elsewhere other than Hudson. My next post will talk more about this library, so stay tuned!

Kohsuke Kawaguchi, the creator of Hudson, wrote the majority of Hudson’s core single-handedly. He is a founder of InfraDNA, which provides products, services, and support for Hudson.

Share

Trends in Testing: Continuous Integration, Mobility, Open Source, Cloud

May 10th, 2010 by The Sauce Labs Team

Functional testing is the automation of web app testing across several platforms or browsers. In the past, functional testing was less common due to long development cycles and the lack of multiple browser options. It was done primarily in-house via proprietary software like Rational or QuickTestPro.

Today, there are four trends converging to radically change how organizations test web applications:

  1. Continuous integration – An increased emphasis on performance of the software through the end customer’s perspective is leading agile development. Gartner predicts that by 2012 agile development methodologies will be used by 80 percent of all software development projects. Teams are shifting away from rigid quality control to quality that is demonstrable to the end user. With this shift, comes a need for quick, simple and automated testing tools.
  2. Open source – Open source tools, like Selenium, are viable for functional testing. Today’s development environment has dramatically changed and a lot more people are contributing than ever before. The major advantages of open-source are speed, time to market, time to value, and the ability to reach and grow a developer and user community. In fact, Gartner predicts that by 2012, 80 percent of all commercial software will include elements of open source technology.
  3. Cloud computing – Testing in the cloud is an affordable and scalable alternative to testing behind a firewall. It is expensive and time consuming to maintain on site test infrastructures that cover a vast number of browsers (and versions) of operating systems in several languages. The increase in the adoption of cloud computing creates an opportunity to leverage the space for functional testing. We believe the life cycle for cloud-based applications will look different in the future. New solutions that are cloud based will support cloud-base applications.
  4. Mobility – Websites need to support multiple browsers such as Internet Explorer, Firefox, Chrome, Rockmelt, Opera on various platforms including Windows and Mac. Smartphones only add to this “Browser War 2.0” battle with the plethora of mobile operating systems. These various environments make the infrastructure required more complex and the need for functional testing more relevant that ever before.

Are there other trends beyond continuous integration, mobility, open source and the cloud that will change how organizations test their web applications?

Share

Continuous Try Server Integration

December 24th, 2009 by John Dunham

Collin Jackson of Betable talks about their successful use of “Continuous Try Server Integration” using Selenium together with Sauce OnDemand cloud-hosted Selenium service.

Share

Continuous Deployment—the video

September 27th, 2009 by John Dunham

On 22-Sep-09, at the Sauce Labs-sponsored SF Selenium Meetup held at the Hacker Dojo in Mountain View, CA, Timothy Fitz knocked the socks off of an assembled group of 70+ web developers, presenting about IMVU’s legendary Continuous Deployment system. As promised we recorded the presentation for the benefit of those who could not attend. Enjoy:



To make it easier to follow along the slides with the video, here they are separately:

Creative Commons License
Continuous Deployment by Timothy Fitz is licensed under a Creative Commons Attribution-No Derivative Works 3.0 United States License.

Share