Survey Says: Agile and Continuous Integration Have Altered How Dev Approaches Testing

Today, software professionals who are responsible for ensuring web and mobile application quality are bombarded with a myriad of challenges. They are dealing with the sheer volume of browser versions and device types that are employed by users, the pressure to deliver their apps to market faster, and many are undergoing structural and organizational shifts such as moving towards DevOps and the goal of continuous delivery. These obstacles are challenging to say the least. These complex problems bring up some important questions:

• How are today’s development teams balancing complex user environments with limited testing resources?
• Are teams adopting new testing technologies like cloud-based services and simulated environments?
• Have modern development approaches made a difference?

A recent independent, global survey of 504 developers titled Web and Mobile Testing Trends, conducted by Dimensional Research serves to provide some clarity and answers around these questions and more.

The study first shows that a shift to continuous integration (CI) and other agile methodologies is driving a massive change in the way that development and testing professionals approach testing. As more development teams adopt agile methodologies, it forces organizations to re-think how they approach testing in a context of greatly increased demand for test suite reliability and speed.

Another finding is that as continuous integration and other agile methodologies are introduced into the environment, developers say they are testing earlier and more often – ultimately leading to better quality software. Additional key findings from the report include:

Modern software development approaches change testing

• 94 percent of respondents say adopting CI or other agile methodologies have changed the way they test
• 69 percent do more automated testing
• 61 percent test earlier in the development cycle
• CI and agile development teams do more of all types of testing with the exception of manual testing

Mobile testing creates unique challenges

• Mobile testing embraces both web applications used in mobile browsers (89 percent) as well as native or hybrid mobile apps (53 percent)
• The survey revealed no standard approach for testing diverse mobile environments
• 29 percent use only simulators or emulators
• 37 percent use only real devices
• 34 percent use a combination of simulators and real devices

Cross-browser testing remains vital

• 80 percent say cross browser testing is “very important”
• 43 percent test only the most recent version of each browser
• 76 percent do cross browser testing across mobile and desktop browsers
• 79 percent say testing more types or versions of browsers would increase quality

Other trends impacting testing

• 43 percent run unit or functional tests for web and mobile in the cloud
• Only 45 percent have fully ended support for Windows XP

There is no denying that each and every technology change impacts both testing technology and processes, and development teams have little control over the devices and browser types that are being used to access today’s applications. As Agile, continuous integration and other methodologies are introduced into environments we can assume that today’s developers are testing earlier and more often – ultimately leading to better quality software.

To access a copy of the report, visit: http://bit.ly/1JVxDwq

About the Author

Jim Cerna is CEO of Sauce Labs, which provides the world’s largest automation cloud for testing web and native mobile apps. Follow Sauce Labs on Twitter at https://twitter.com/saucelabs