N26 Leaves Manual Testing Behind to Reduce Testing Time, Increase Coverage, and Boost User Confidence

Founded in Berlin in 2013, mobile bank N26 has redesigned banking for the smartphone, making it easy, fast (opening an account takes eight minutes), transparent, and contemporary. N26 launched in 2018, and is available in over 25 countries within the Eurozone. More than 7 million customers manage overdrafts, withdraw cash in retail stores, send money abroad, invest or save, use an insurance service, or apply for consumer credits within the N26 app.

Because its product is entirely consumed on mobile, two of N26’s core assets are its iOS and Android codebases. A few years ago, N26 had a small QA team manually testing new app features on ten different phone models, and five different operating system versions – all of which was time consuming. 

“We wanted to automate our testing, especially as our codebase and feature set were growing,” says Wojna. We knew that manual testing would become more time consuming and possibly slow down our release process. Different parts of the app are written in different languages making testing more complex as well. 

Wojna and her colleagues wanted to use the Appium test automation framework for

a proof of concept, because it allows the team to write test scripts in their chosen languages and can be run across both iOS and Android to simplify testing. Because it’s a popular framework, the team knew that the documentation and user community would help them if they had problems during the integration.

Instead of running Appium tests on local devices, N26 chose Sauce Labs, which provides the framework on virtual devices in the cloud. “We had a choice: build an in-house device farm or look to the cloud. We’re a forward-thinking company, so using the cloud for mobile app testing was a logical choice,” says Wojna.

Today, the team runs one automated Android regression test a day, and they are in the process of fully integrating iOS testing. N26 has recently incorporated automated testing with one of the open source automation servers, which helps with the development team’s move to continuous integration (CI). “We’ve been very open and worked with our developers to make Sauce Labs part of our CI pipeline,” says Wojna. “This has made the transition to Sauce Labs easier and raised the profile of automated testing within N26.”

N26 now tests its app on up to 100 devices. “The main benefits to our business are continuous releases and happier customers” says Wojna. “Our automated testing framework on Sauce Labs identifies more bugs before release, meaning a better customer experience.”

For QA and development teams, the benefits are the time – and money – they save on complex manual tests, and greater confidence in the results of tests. “Using Sauce Labs saves us a lot of time,” says Wojna. “We can now focus more on creative work and understand our product better during exploratory testing, for example. Our goal is to give QA engineers in each of our teams the ability to test the features built by that team in an automated way. The team is also working to get parallel tests running in order to save even more time. We’re happy to be on that journey with Sauce Labs.”