Okay, honesty time: browser testing sucks. It's stuck in the stone age. Browser tests are great when they're working well, but they're way too hard to get right, way too expensive to maintain, and just overall not a lot of fun. That's because existing tools lead us to build brittle, sequential tests that can be clobbered by tiny application tweaks. What starts as straightforward test creation ends up with data, actions, and selectors thrown together in a big mess. We paper over the mess with abstractions like Cucumber and Capybara, but we haven't addressed the fundamental issues. We're the cavemen of browser tests. Fundamentally, these are the problems with browser tests as they're written today:
Getting started is hard. It's important to structure your tests in a way that will work out well long-term, but guidance is scarce.
Browser tests are brittle. Small changes in the DOM, in application performance, or in other tests often generate a lot of menial labor to repair tests. A single DOM change in your application can easily cascade into hundreds of broken tests — and cost staff days to fix them.
Browser tests are slow. The solution is to run many tests in parallel, but current testing frameworks lead us to write tests that don't parallelize well.
All this makes writing and maintaining tests an expensive, and more importantly unpleasant activity. We can do better. It's time for a new way. As an industry, we know how to do this a lot better than we're managing to in most cases. It's time to accept a standard, move on, and get things done. Test::Right aims to be that standard. The ultimate goal of Test::Right is to make it impossible to write bad tests. That's right, read it again. Impossible to write bad tests. That's a serious goal, but we're serious about it. Are we there yet? No. But we're a whole hell of a lot closer than anything popular out there today. Our philosophy in creating Test::Right is to directly address the problems with browser testing from the start:
To make it easy to get started, test authors need an opinionated framework that chooses a thoughtful structure for tests based on our extensive experience working with large test suites.
To make browser tests less brittle, we have decoupled intentions from mechanics. Stable intentions like "log in" and "check account status" are disentangled from volatile mechanisms like "click the DOM element identified by this CSS selector". We isolate the parts of tests that change frequently in a single place instead of scattering and duplicating them throughout tests, making tests robust and maintenance easy.
To make browser test suites run faster, we parallelize your tests from day one. Your builds will always be fast, and because every build checks that your tests run in parallel, you'll never develop performance problems that will bite you later.
Test::Right is an idea whose time has come, born of dozens of on-site integrations of our service in household-name companies and some not-so-household name places. We as a small team have seen more browser test badness than any sane person should have to, and the tools are at fault. Test::Right is designed to address the mainstream case we've had the privilege to experience over the past 16 months. We're releasing it as a work in progress, but it already has a ton of useful functionality. We've put a stake in the ground with a whole slew of opinions that are sure to get a discussion started. And that's really where we need you. We built Test::Right by borrowing concepts from the Page Object model, but went even further by enforcing clean separation of concerns. With Test::Right, your test cases are defined in terms of actions and properties on widgets. A widget is a piece of functionality present on one or more pages of your application. A single page can have many widgets, and multiple copies of a widget may appear on the same page. Tests are grouped by feature, and tests don't have direct access to the underlying Selenium object.
Widgets bring the Page Object concept into the Ajax age and decouple tests from DOM, making it far easier to adapt to frontend tweaks. They give your tests an application-specific vocabulary that maps directly to what users do with your app, and let tests make assertions about high-level aspects of your app, instead of low-level DOM state.
Automatic spin asserts ensure that your assertions work as you expect them to, even with Ajax.
Out-of-the-box parallelization gives you speed and scalability from day one.
Random test ordering prevents you from writing bad tests that serially build up state, depend heavily on one another, and can't be parallelized.
Data factory generates isolated test data so that tests don't clobber each other.
Test::Right is in Ruby because it seemed like a good place to start. We believe these ideas make as much sense in any other language. Whether or not you use Test::Right, we want you to come away from it with ideas and opinions. The browser testing community needs to wake up and start writing good tests, and with Test::Right we're going to make that happen. And I, personally, look forward to hashing out the future of browser automation with you. Check out the Test::Right source code on GitHub to get started. Let us know what you think, and enjoy!
