We love automated browser tests, but we *really* love fast builds. The faster the better. So when a build that I'm waiting for starts taking over 5-10 minutes, all I can think to myself is... "Hurry the hell up!" And we all go through it, even here at Sauce. So here are a few things we've found to really help speed up Selenium testing:
The more tests you can run in parallel, the faster your build can finish. So when writing your tests, always ask yourself, "Can I run this test by itself and have it pass?" If you think to yourself, "No... another test has to run *before* for this one to pass," then stop. Put the shovel down, and start climbing your way out of that hole. Tests that rely on previous tests cause random, weird bugs to pop up, are hard to run when you want to reproduce those bugs, and are just slow. So keep your tests independent. It's how they were meant to be anyway.
When none of your tests are inter-dependent, that means they can all be run in parallel, and in theory, your entire build should only take as long as your single longest test (in practice, it's a bit longer, but not too much). Which brings us to the next point...
2) Short Tests
Write your tests to only test a single tiny bit of functionality. Keep the number of steps in your tests under 20. All the parallelism in the world won't save you from long-running tests, because a single test can't be parallelized. The best written tests we've seen generally run in under 10 seconds. Yeah, I know. That's short. If you're wondering how that's possible, check out step #3, below.
A pleasant side-effect of this is that when a build fails, you'll be able to narrow it down to a very specific area point and fix it all the faster.
3) Pre-populate the cookies
This is a really nice trick.
But this poses a bit of a problem with shorter builds, because now each browser session has to go through a login process, and possibly a few more steps depending on your application logic, just to get to where you want to test. Previously, I simply recommended abstracting the login process into a separate method so that the tests looked cleaner, but when you're waiting for the build to finish, that's not good enough. You want them clean *and* fast. One way to get this is to set the cookie via Selenium after the browser starts up.
Log in to your user via server-side logic (e.g., generate a row in your sessions table, or however you have it setup locally), generate a cookie for that login session, and push it into the browser via Selenium. Bam. You're done logging in, and your test can continue on as though it had already spent the 5 seconds logging in and the 10 seconds adding items to the shopping cart, all so you can just test the "confirm purchase" button works on your site as expected in IE7.
This isn't to say that you shouldn't test out your login form or your "add fuzzy critter to cart" logic via browser tests. Because you should. But you shouldn't do it for every test. Test it once, test it good, and leave it at that.
This works on anything that relies on state to be built up. Logins, Shopping Carts, Playlists, etc. You get a clean browser for every tests, and your tests are speedy. And you know what? Even Fred'll love it. And that - in and of itself - is worth it.
Got any tips on how you speed up your Selenium builds? Take us to school and share 'em. We're all ears! Oh, and for Fred's sake...
|\ _,,,---,,_ /,`.-'`' -. ;-;;,_ |,4- ) )-,_..;\ ( `'-' '---''(_/--' `-'\_)
SAUCE LABS - SELENIUM AND APPIUM TESTING IN THE CLOUD