Dr. Testlove or: How I Learned to Stop Worrying and Love Automated Testing*

This is a guest post by Brent McNish, the co-founder and CTO of Deliberator. Deliberator is a joint customer of Sauce Labs and Solano Labs (makers of Tddium).

* with apologies to Stanley Kubrick

 Pre-testoric

The image of the head-down coder hacking away Tasmanian Devil-like, paying lip service to writing tests, is a thankfully less and less accurate cliché these days. But that wasn't always the case. These guys (and girls) used to be everywhere. They didn't get into development to do testing! Look, the code works! Job done. Next feature, bring it on!

Yes, once upon a time the “who needs automated tests” dinosaurs ruled the earth. And I was one of them.

I began my web development career in a large IT consultancy. We had testers, and test managers, and elaborate test plans. They would catch the bugs because it wasn't a developers job to test. Then I left to co-found my first (bootstrapped) startup, as the sole developer, with a single co-founder. And in Bootstrapped-Startup-Land, you don't have testers, and test managers, and test plans. There is just you. And your code. And every line of code you write, someone, somewhere, is also writing a line of code, and they might have the same idea as you. And they're going to launch their feature before yours. And they're going to beat you. So each line of code is precious. And you don't want to "waste" it on a test. So I still wasn't interested in testing. When each feature was completed, I manually tested it, then my co-founder tested it. Then I fixed any bugs. Then we both tested it again. Then, when it worked, I moved on to the next feature. Then things that had worked would break. So I'd go back and make them work, then move on. Then, later, they broke again. A mantra began in my head, quietly at first, then with increasing volume: "Write some tests, write some tests..." But where to find the time with all this bug fixing to do? Yes, that way madness lies.

(Unit)ed we stand

So I began writing tests. This was a baptism of fire as, of course, there was now a large backlog of untested functionality to tackle. But I gritted my teeth, girded my loins (whatever that involves...), dove in and began writing unit tests. And, you know, a funny thing happened. Slowly, very slowly, assertion by assertion, I learned to love testing. I began to take actual pleasure (pleasure! imagine that!) in crafting a test and then seeing the little green icon in my IDE ping to life when it passed. Knowing that my new feature was fit to go live. And more importantly, that I hadn't broken something else in the process. So, this was job done right? Sure, this didn't test the front-end. But that's what humans are for right? Wrong. The extra confidence and speed I gained from the unit tests simply seemed to manifest itself in more front-end bugs. Dammit!

Seleniummmm

Then I discovered Selenium. This was Selenium 1 (aka Selenium RC) so not the most stable and robust framework ever. We would often see a test fail then pass immediately after without any change to the code or data in between. Hmmm......

Even so, automating browser tests seemed like magic. It was mesmerizing to watch the tests running. A never-tiring invisible hand filling in form fields and clicking buttons. (Ok, maybe I'm just easily mesmerized). It was fortunate that I found watching the tests so entertaining because boy were they s-l-o-w. A full run would take around 90 minutes. It also sent the CPU fan on my laptop crazy and made using it for any other purpose at the same time a painful ordeal. The upshot of this was that I didn't run the Selenium tests very often. Which meant they grew more and more out of date. Which meant that I was even less likely to run them. So we ended up falling back on manual browser testing again. Doh!

Hot Sauce

But, wait, what's that sound? Enter stage left, our hero on a white horse..... it's Sauce Labs! Yes, I remember distinctly the day I came across the Sauce Labs website. I instantly Skyped my co-founder “Praise the Lord!” or words to that effect. “Selenium is re-born!” And for us it really was.

After a very small amount of painless integration, there they were, our browser tests running in the cloud. Sweet! The test sweet, ahem suite, still took a long time to run but it was now fire and forget. Just kick off the tests and get on with your normal business. (Although I did kinda miss being able to fry eggs on my Macbook when the tests were running locally. Those were damn good eggs!). Almost as big a deal as being able to run our tests in the cloud was that we now had a complete record of every Selenium test run, including a video of the test running! Add to this the ability to do ad-hoc cross browser/OS testing with Sauce Manual and test against our local dev build with Sauce Connect, and it's fair to say we were pretty ecstatic! This was by far the best situation we'd been in test-wise and the quality of the code showed it.

Carpe Tddium

But, you know, I'm hard to please. My two biggest remaining niggles for me were the speed of the Selenium test suite execution, and the fact that our unit and Selenium test results weren't integrated. I'd come to accept these limitations, until... What's this, the rumble of horses hooves again? Here comes the second hero of our piece, Tddium, charging into the fray! The claim of Tddium to completely lift the testing burden - unit and Selenium testing - into the cloud, and run both in parallel blew me away. To the extent that I was dubious as to how well it would work. But the answer? Very well indeed! Now add in Tddium's Github integration and Continuous Integration support and.... wow... just....wow. I am currently running 8 Tddium workers in parallel and the run-time for my complete suite of unit and Selenium tests is down from around 2 hours to 15 minutes. This has been a game-changer in my development routine. I'm now much more ready take risks and try stuff, knowing I can get such quick and comprehensive test feedback.

Continuous Inspiration

So yes, my conversion is now complete. From throwing code over the wall to 'those tester people', to being forced by necessity to slowly embrace automated testing, to now realising the full potential of automation with Sauce and Tddium, it's been quite a ride. And it's not finished yet. I still know that, someone, somewhere in another startup is still writing that line of code, and they might have the same idea as me. But now I'm not worried that they're going to launch their feature before us. And they're not going to beat us. Unless they're using Sauce and Tddium too.... Dammit! --- Brent McNish is the co-founder and CTO of Deliberator, a new social network for ideas. Deliberator brings people together to create, debate and propagate solutions to the complex problems of the day. Join the debate at www.deliberator.com