Continuous Testing in the Retail Industry

Mar 29, 2018

In today's retail marketplace, digital channels hold a prominent and often central position. Modern software practices emphasize the need for rapid development and deployment. The retail world is no exception to this need for speed. Continuous testing is a great strategy for getting increasing velocity.

Continuous Testing for Retail

Having worked on both sides of the engineering and management coin, I know the difficulties in arguing for the allocation of resources to automated testing and setting up continuous testing. To the non-technical manager, it delivers no public facing feature, and no improvement that seems to retain or convert customers. If software isn't your product, then why adopt the practices of the software industry?

Four ideas I emphasize when explaining the benefits of continuous testing

There are several reasons for continuous testing:

  • Retail software needs to constantly evolve
  • The customer experience needs to be guaranteed
  • Human testing isn't enough
  • Time and resource savings from early bug detection

Let’s explore these in order to understand where continuous testing fits within the retail industry.

1. Retail software needs to constantly evolve

One of the big differences between software and more traditional products is that software is constantly changing. Like a shark, software development has to keep moving forward or the product will die. Traditional products have high upfront costs to ensure their quality. All of the testing happens before the product is released. You can't change the design of the shoe that is already in the customer's closet. With the majority of software products, we are not only able but obligated to continue to evolve the product after its release. One of the great business advantages of this is that we can very quickly develop a minimum viable product that sales and marketing can get in front of the public for feedback. We can and do make major changes after release, whether it's a redesigned UI, an improvement to our machine learning algorithm, or vast improvements in the ability to scale. Imagine if a car manufacturer called you and told you they have completely redesigned the engine for your car—not for new models in the future, but the car you've already been driving for two years.

The demands of retail require this ability to change as much as any industry. The nature of the industry is to be time-sensitive. There are holidays, sales, and new product lines that can't wait for long development cycles. Think about the speed with which a brick and mortar store changes its atmosphere to reflect marketing needs, whether it's a new window display, a different banner hanging outside or new artwork inside. Digital channels need to evolve just as quickly—a Christmas redesign of your app which isn't ready for release until January is both wasted effort and lost opportunity.

Increasing the velocity of releases is one of the benefits of continuous testing. Once your test system is set up to run tests with every change, your developers can push changes faster. They get immediate feedback on whatever new changes they introduce, rather than waiting for human QA.

2. The customer experience needs to be guaranteed

One of the advantages of digital channels in retail is the ease with which a customer can access your products. They expect to be able to shop anytime, anywhere, and this is what we strive to give them. Hand-in-hand with this ease of use is the flexibility to move on quickly to a competitor. We should expect them to be fickle if we don't deliver an excellent experience. Slow apps, bugs, downtime, and security concerns can drive them away and leave them with a lasting bad impression.

With continuous testing, we can be constantly checking our software for regressions, bugs, and performance problems. Catching bugs early in our process reduces the chance they will make it into production. The confidence that continuous testing brings also allows for faster experimentation, and a tighter feedback loop with the customer as release velocity increases. Overall, a customer experience with fewer bugs and performance and security issues coupled with better reaction to customer feedback is part of the promise of continuous testing.

3. Human testing isn't enough

Software systems are like an old sweater—pulling a thread in one sleeve could lead to a hole in the collar. Changes in software can have unforeseen side effects. For tests to truly bring us protection, they must be comprehensive. They must be repeated often. The nature of this kind of work is repetitive and prone to error when performed by humans. It’s better to use software to our advantage by coding repetitive, boring tasks into test suites which we can run automatically. This has the added advantage of freeing up human QA for more human-based testing—testing user experience. So with continuous testing we can both improve the quality of our test runs and free up humans to improve the customer experience.

4. Time and resource savings from early bug detection

One of the advantages of continuous testing is the saving of time and resources. In historical software development, the bulk of QA happens once there is a release candidate. The release must then be held up while corrections are made—corrections which may be difficult or impossible due to multiple, concurrent feature changes. By running tests earlier and often, we can detect bugs before bundling new features together. The developer can see that changes are needed, or that a feature needs to be redesigned before investments have been made in other features. Detecting bugs early means that they can be corrected with less effort and without wasting the time of other teams.

Modern retail depends on digital channels, and requires these channels to be reliable. It also requires a constantly improving customer experience, or your customers will go elsewhere. Continuous testing improves the ability to deliver both.

Kennedy Behrman has over 20 years of experience in software creation and management. He has worked in numerous startups, movie studios and in international economic development projects.

Written by

Kennedy Behrman


Continuous testing