How to Lose Races and Win at Selenium
Selenium races your browser
Selenium tests sometime fail for no reason. You can eliminate most of their random failures with a special kind of assertion, the spin assert. The problem is that Selenium is overeager, and it needs to chill out and wait.
Selenium tests often fail because they're too fast. Where a user might wait for a page to load for a few seconds and then click on a link, Selenium will interact with a page at the speed of code, before the page is ready. The way to fix this is to have Selenium repeat its actions and assertions until they work. If you don't, Selenium races your browser.
You need Selenium to lose the race
The way to solve that is not telling Selenium to pause for a fixed number of seconds. Pauses have two problems. They make your tests slower than they need to be, and they don't really fix the problem, just make it less frequent. The best solution is to have Selenium keep trying until it works. If your test is trying to assert that clicking on an "email settings" button pops up an email settings dialog, Selenium should repeat a "click the button -> is the status dialog visible?" loop until the status dialog is visible.
Here's how to do this
The solution is spin asserts. At Sauce Labs, when we want to verify that some text appears on a page, we use this function:
def wait_for_text_present(self, text, msg=None): msg = msg or " waiting for text %s to appear" % text assertion = lambda: self.selenium.is_text_present(text) self.spin_assert(msg, assertion)
Do you see it calling a function named
spin_assert? That function is the key.
spin_assert retries the passed-in test function, in this case a lambda expression, over and over until it works. Here's what
spin_assert looks like, minus some bells and whistles:
def spin_assert(self, msg, assertion): for i in xrange(60): try: self.assertTrue(assertion()) return except Exception, e: pass sleep(1) self.fail(msg)
In this case,
wait_for_text_present will repeatedly ask the Selenium server whether the text is present until it shows up. It could do several things in a loop, like click a button and then ask whether a popup has appeared. Almost all the asserts in our internal Selenium testing are a wrap around a call to
spin_assert, and they're all way more reliable than they were before the switch.
- Accessibility Testing
- Appium Resources
- Best Practices
- Continuous Delivery
- Continuous Integration
- Continuous Testing
- Cross Browser Testing
- Guest Blog Posts
- Load Testing
- Mobile Development & Testing
- News & Product Updates
- Open Sauce
- Open Source
- Performance Testing
- Product Updates
- Quality Assurance
- Quality Engineering
- Sauce Product Info
- Security Testing
- Selenium Resources
- Software Development & Testing
- The Story of Sauce