Imagine a misshapen old cardboard box sitting on a desk in an office somewhere, intended to hold testing devices.
Now imagine how many devices this box will accumulate over two years given the pace at which new hardware hits the market. It could be anywhere from 20 to 30 devices, with 10 functioning charging cables.
Now picture someone reporting an issue with an iPhone 7 – an incredibly outdated device – and forcing you to dig around in said box. You successfully find an iPhone 7, only to realize it hasn’t been charged in a year. Productivity? Oh, it’s completely down the drain at this point. The battery’s probably worn out, as well. To add to this, the operating system is also outdated, so in addition to all the time you’ve already lost, you’ve also got to install the latest version of the OS.
To put this all into perspective, take a long hard look at the diagram below which illustrates the costly and unsustainable evolutionary pace of the device and operating system landscape.
If you’re finding this visual overwhelming, it's because it probably is. Each of the dots above represents the number of devices and operating systems (in beta and generally available) your team will be responsible for keeping track of. Imagine how many hours it would require to keep up with all of this realistically.
If your organization manages all of your testing infrastructure in-house through device carts, device farms, or in-house devices, that’s putting a lot of undue strain on your development and QA teams.
Hardware costs are only the tip of the iceberg. There are a multitude of hidden overhead costs to consider as well.
“It’s easy to say 'I’ll just buy a few devices, keep them updated, and roll them out'. But what’s often overlooked is the necessity of a testing infrastructure, like a staging area, where you test your updates before releasing them.”
Diego Molina
What are the key challenges associated with on-premises device infrastructure?
Procurement and shipping costs - Sourcing devices from multiple manufacturers - Shipping devices to offices and multiple remote locations - Insurance and carrier plan for each device | Maintenance costs - Dedicated person-hours for new OS and device updates - Managing different app versions - Periodic device pool replacement - Cost for lost devices breakage, battery repairs
|
Operations costs - Dedicated person-hours for support of on-premise devices - Device cleaning after use (Removing temp files, uninstalling apps, etc.) - Device charging and ensuring ready-to-use -Device lock-outs, crashes, etc.
| Security costs - Managing device security - Legal and compliance fees
|
Let the experts handle the legwork
Engineers want to build things and an on-premise device infrastructure might be a fun project. However, maintenance and constant updates turn a fun project into a nightmare. Frustration arises when the focus on delivering value is lost. QAs and developers want to innovate and release as fast as possible. QAs and developers are internal customers who deserve the best device infrastructure experience. When customers don’t get the best experience, they will look for a better experience elsewhere. It’s very easy to get caught up in innovating and releasing fast. Investing time in maintaining your infrastructure is investing time in your people.
At Sauce Labs, we have people in a data center responsible for maintaining everything, even down to procuring the latest devices. When we think about having a testing infrastructure we want to have a place where we can guarantee that the experience we provide to our customers is the best one.
If you’d like to learn more about the market pressures, challenges, tradeoffs, solutions for on-premises testing infrastructure versus using the cloud, and how to maximize ROI for your mobile testing strategies, catch up on Part I of our Mobile Testing Masterclass led by a couple of our experts.
