When it comes to automating mobile tests, it is a good practice to use both virtual and real devices to optimize costs and get your apps to market faster. It’s obvious what a “real device” is: the physical phone or tablet of a certain make, with a certain mobile OS installed. What, then, is a “virtual device”? This is the umbrella term for iOS simulators and Android emulators. One is called “simulator” and the other “emulator” for the technical reason that iOS simulators execute mobile apps within macOS by simulating the iOS environment, whereas an Android emulator actually implements the Android OS as a VM, regardless of the host environment. The terms emulator and simulator often get used interchangeably.
What is an emulator?
An emulator, as the term suggests, emulates the device software and hardware on a desktop PC, or as part of a cloud testing platform. It recreates the device (hardware and software) on a host machine. This re-implementation of the mobile software is typically written in a machine-level assembly language, an example is the Android (SDK) emulator.
A simulator, on the other hand, delivers a replica of a phone’s user interface, and does not represent its hardware. It does not run the real device OS; rather, it’s a partial re-implementation of the operating system written in a high-level language. The iOS simulator for Apple devices is one such example.
Mobile emulators versus real devices
Mobile emulators are software-driven and therefore much faster to provision than real devices. Additionally, they enable parallel testing, and test automation via external frameworks like Appium. Selenium revolutionized the world of web app testing by pioneering browser-based test automation. Today, Appium is its counterpart for mobile app testing. Appium uses the same WebDriver API that powers Selenium, and enables automation of native, hybrid, and mobile web apps. This brings huge improvements in the speed of tests for organizations coming from the manual world of testing on real devices.
However, QA teams that start to use only mobile emulators may swing to the other extreme of stopping all testing on real devices. While this speeds up the testing process, it comes with a critical drawback — emulators can’t fully replicate device hardware. This makes it difficult to test against real-world scenarios using a mobile emulator. Issues related to the kernel code, the amount of memory on a device, the Wi-Fi chip, and other device-specific features can’t be replicated on an emulator. It’s not enough to test on emulators alone. Real devices are an important part of the QA process.
Emulators & real devices - better together
Today, organizations fall under one of two extremes. They either rely only on real devices, or only on mobile emulators for their QA. Some organizations test exclusively on real devices with the assumption that they aren’t compromising on the quality of their tests, while other organizations test exclusively on emulators because they are faster than real devices, easier to maintain, and cost much less. However, both of these extremes are a compromise. Real devices have drawbacks in terms of scalability and cost. Though emulators are an improvement on real devices, they are unable to deliver a real-world testing environment.
The ideal QA strategy employs a mix of mobile emulators and real devices. This option addresses the scalability and cost inefficiencies that come with real devices, while retaining the ability to test under real usage conditions. It provides the best of both worlds.