The build and deploy cycle for web software is fast. Teams that take a DevOps approach might be able to move new code from their local branch into a clean test environment in a matter of minutes through containers and build-deploy tools like Puppet. Mobile software isn't quite there yet. Developers write code, then send it off to CI to be built, and at that point they have lost control of where the new build goes.
How does the mobile build and deploy cycle work, and what affect does that have on testing?
When testers or developers push a new build of their product to a test environment, there is no questions of who has access. Everyone that knows the right URL, and has an account is now using the latest version. Mobile devices on the other hand are independent from the build cycle. You might have five test devices in-house, each running a different build. When a new build comes out, each mobile device needs some way to get the latest build. Hockeyapp and Testflight are popular ways to make this happen.
After a developer creates a new build, they need to make that build accessible to test devices, often by adjusting configuration in a distribution tool. For people testing, this is an opt in process. That's different that normal web application testing, where everyone automatically gets the updates to the test or staging servers, like it or not. This does two things in terms of testing; distribution tools insert wait times, and make developer testing using emulators and simulators even more important.
A few years ago I was working on an iPhone app being distributed through Hockeyapp. The developers wrote some code, and then pushed a new build. I opened Hockeyapp, downloaded the latest build, and started testing. Right away some things were just wrong. This created extra back and forth between dev and test that took more than a day to resolve.
That extended delay between dev and test makes the quality of builds important; there just isn't time to run through the loop several times because of obvious bugs.
As of today, there is no continuous delivery model for mobile software. As far as I know, there is no way to write a couple lines of code on a special branch, complement those changes with unit testing, service tests, and pairing, and then deploy to production after seeing a green bar in Continuous Integration. There are little, or big, unavoidable pauses all along the way. The Google Play Store and iTunes represent the last pause in the mobile deployment cycle. Once the app is submitted, it can take up to a week for Apple to complete their review process and make the new build available in iTunes. The process for getting into the Google Play Store can be a little faster. Still, if an important bug is discovered in production, we have a problem - days of delay mandated by the distributor.
Luckily, there are a couple of ways to mitigate this risk. Developers can write new code and test in an emulator to instantly get a feel for how the product is coming together. Once there is a viable build, the team can get device and platform coverage using emulators. And at some point, they can get deeper test coverage by using real devices in their hands or through private and public device clouds. Actual deployment to the app stores can be rolled out in stages. If a problem happens in production, this gives the development group an opportunity to kill the roll out before too many people are affected by the new bug. In some cases, the team may be able to set up configuration flags, to turn off a feature at the server, instead of requiring a rollout to a store.
The mobile build and deploy process has pauses and wait cycles inserted that we just cannot get rid of at this point. There are little breaks in flow between build and deploy to test devices. And then there is a larger pause when a company is ready to go from development into production. The stakes are higher each time a new build is moved, so getting things right, or as close to right as possible is much more important. Making use of environment tools like emulators, simulators, and device clouds can certainly help.