After we ran our billionth test, we thought long and hard about what we learned over the years. We also asked customers for tips they’ve gleaned working with us on their automated testing journey.
Mike Millgate, automation architect at Optum, says his #1 tip is “Don’t do testing, just ship it. It speeds up the whole CI/CD pipeline!” Mike, of course, is kidding here. However, he has offered up some great tips and best practices for anyone who is automating their tests and/or using Sauce Labs:
Fail, and fail often. Learn from your mistakes to make it better.
Run small, self-sufficient tests.
Execute tests in random order.
Include multiple levels of parallelization.
Tweak the desired_capabilities used for the Sauce Labs sessions to meet your minimum needs.
Watch total execution time of each tests, configure a low max duration setting, refactor where necessary.
Make sure the team communicates and feels ownership equally. QUALITY is a TEAM effort!
Track stats over time.
Use Sauce Labs Support.
Watch video playbacks to figure out failure anomalies.
Stay focused on Sauce Connect Tunnels.
HAVE FUN!
Join Mike at SauceCon, where he will present on “Testing @ the Speed of Concurrency.” He will talk further about the journey into cloud-based testing to show what worked, what didn’t and how to improve as the machines are increasingly doing the work.
