Guest post by Aytekin Tank, co-founder of Interlogy Internet Technologies
Why NOT Automated Selenium Tests In-house? The biggest problem with running Selenium tests is creating and managing many testing environments. Usually if you are running Selenium tests constantly on a machine, it cannot be used by anyone. So in practice, they have to be on dedicated machines. And if your releases depend on the Selenium tests, you have to make sure they are always on and running smoothly. It is a lot of maintenance headache. The other problem with Selenium is that it is slow. It takes Selenium a long time to launch up the browser to do the tests. Unless you have many dedicated machines, the tests have to run sequentially. To speed up our Selenium tests, I looked for ways to run them in parallel. My first approach was to use Amazon EC2, since I am already pretty familiar with it. That turned out to not be a great solution. We still had to maintain many selenium environments. Keeping many EC2 instances always on can be quite expensive. Launching new instances on demand turned out to be too slow for our case.
Final Solution: Sauce OnDemand Then, I found out about Sauce Labs’ Sauce OnDemand service. This is basically an on demand service that lets you run Selenium tests instantly on the cloud. It has many good code samples. So, setting it up was pretty straightforward. The great thing about Sauce OnDemand is that you can choose from many browser setups and run your tests on them instantly. No need to maintain dedicated machines. Unfortunately, there was no documentation about running tests in parallel in PHP. So, we first looked into using PHPUnit’s new parallel processing features: @runTestsInSeparateProcesses and @runInSeparateProcess. They turned out to be too buggy to be usable at this time. So, we hacked together a simple script that runs the tests on the shell as background processes.
A Simple Example I am a big Perl junkie, but since our existing unit and integration tests were written in PHPUnit, we wanted to keep our Selenium tests consistent and write them in PHP. Below, you can find a simple example showing how you can run automated cross-browser Sauce OnDemand tests on PHP. We hacked this together quickly so it is not documented or optimized. But it should give you a good jump-start:
1. BrowserTests.php: Includes actual Selenium tests and simplified for this example with a single test. On our production version, we have all the tests here, since we prefer to save time and run all tests at once on a specific browser. You can create these tests using Selenium IDE or Sauce IDE.
2. allBrowserTests.php: You can run this on the shell then add it to Hudson or any other Continuous Integration tool. Basically, this script runs Selenium tests on the background in parallel. It then checks their log files to see if they finished successfully or failed.
3. SeleniumTestHelper.php: This is the integration script with Sauce Labs. You should change the domain name, Sauce username and access key to your own.
Download all files files here. You must first have phpunit installed to use them. Happy testing! Aytekin Tank is a co-founder of Interlogy Internet Technologies. Their flagship product is JotForm online form builder.