How to Start with Agile Development

Posted Nov 10th, 2020

Agile vs Waterfall

It's been four years since Sauce Labs wrote Beginning Agile Development. In the time since, the concept has become so widespread that most everyone knows what it is - an approach that drives the continuous iteration of software development and testing throughout the lifecycle of a project, with development and testing activities done concurrently.

In this update we work backwards, starting with the most common Agile Software Development practice - Scrum - then exploring the essence of Agile. We'll touch on the waterfall that came before, cover two common change management strategies, and then introduce a third, radical method of Continuous Delivery.

Starting Scrum

Start with a whole team that can cover the entire delivery cycle - from work solid enough to begin development to production. That team has three roles: Team Member, Scrum Master, and Product owner. The team builds some potentially shippable unit every week or two, called a sprint. The sprint is the time people do software development. On top of the usual coding, testing, and analysis, sprint adds four ceremonies. Those are the planning meeting, the daily standup, the retrospective, and the sprint review. The review brings in everyone to talk about what was built; the retrospective is just the team talking about how to do better.

That's it.

If you can do those things, you are doing Scrum. The team has ownership of its work process, the team decides how they are going from refined-enough-to-do-engineering work at the beginning of the period to working software at the end.

Of course, it is always possible to spend three days on requirements, three days on development, three days test/fixing, and have an implementation day at the end. People can work separately on what they are good at. It's also possible for people to stay in their lane, focusing on their specialty and "their work." This may be the easiest way to "do" Scrum. Critics call this mini-waterfall, flaccid scrum, or just "not really Agile." 

Becoming agile

Agile Development is a label for a sort of collection of lightweight methods that became popular in the late 1990's. The basic idea was to push back against the contract-driven approach that had come before. Prior approaches to software development focused on nailing down the contract criteria, getting the plan right, having good standards in place, and have frequent quality gates between the "phases" of requirements, development, and test.

The general consensus was that problems found earlier were less expensive to fix. Thus we need to check every piece of work earlier in the process. Sadly, this often meant finishing a piece of work and scheduling a meeting for a code review for a week from next Tuesday. Until the code was reviewed, it could not be tested.

Scrum, Extreme Programming, and other lightweight methods went the other way. Instead of waiting for review, let's build a system of radical collaboration where people are pairing, mobbing, and sharing their work as they create it. Let's get it from development to test in minutes so no one forgets what we were trying to accomplish. While we're at it, let's create a mechanism to figure out what errors are happening quickly and prevent them in the future. The retrospective and standup are supposed to be that kind of mechanism, but to perform them you need an empowered team.

In 2001, at a conference in Snowbird, Utah, a loose group of people supporting these lightweight methods gave it a name: Agile Software Development. The document they created, The Agile manifesto, may be the single most important software document of this century. The major value it proposed include: 

  • Individuals and interactions over processes and tools

  • Working software over comprehensive documentation

  • Customer collaboration over contract negotiation

  • Responding to change over following a plan”

The manifesto also includes twelve principles, such as delivering working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. It suggests business people and developer work together daily, that working software is the primary measure of progress, and implies that welcoming changing requirements can be a competitive advantage.

Looking at the manifesto a different way, it could be summarized as:

Business is not a process of moving things from one inbox to another. Instead, software enables the customer to do something that adds value. So we'll build something quickly, get it in the hands of the users, and get feedback. The most effective way to do that is to do requirements, development, testing, and perhaps even deployment as interleaved activities in small batches.

Now that we understand the "heart" of Agile Software Development, and how Scrum is supposed to enable that, the criticisms make sense.

That leaves the question of how to get started.

Three Approaches

  1. Put people in a room, teach them Scrum, tell them to go do Scrum. One term for this "Shock therapy." The results of this are often the mini-waterfalls and flaccid Scrum discussed above. A more mild version of this makes the change a team at a time. This is a smaller, more bite-sized change.

  2. Make the change a piece at a time. Change from quarterly releases to monthly. Move from requirements to stories. Add a daily standup and invite customers to attend. In theory, this sounds like the "best" approach. In practice, it can be very difficult to create and sustain consensus on what to change next.

  3. Create a continuous delivery pipeline where every push to version control runs through a barrage of automated checks. If possible, use configuration flags to push that change directly to production if everything passes, but only accessible to employees for testing. 

Of the three, it is the third option which really enables the team to eliminate the distinctions between the analysis, development, test and deployment "phase", and collapsing them into a feedback loop that might last, perhaps, a half day.

Exactly how to do that will vary by team, but that is where a second problem emerges - how to structure the teams. Classic software development organized teams around applications, business units, or, at worst, functional specialty. A team could "do" scrum and be capable of delivering work every week (or every day) but have an external dependency of a different team. Organizing by specialty prevents real team cohesion, as the analysts are structured to compete with the developers and testers. The modern Agile answer is to organize around features, giving the team everything they need to deploy.

Here's how.

Create a Feature Team

 A feature team “is a long-lived, cross-functional, cross-component team that completes many end-to-end customer features – one by one.”

This means that the team is composed by people that have different roles and that are able to produce a feature from the very beginning to the very end, without having to interact with other teams.

Team Size.  If the company adopts a Scrum approach, the team will have usually 7 members (+/- 2 members), this is small enough for close collaboration but large enough to have a wide variety of skills.

Benefits.  The purpose of a feature team is to deliver the best possible value to the customer, because the Agile is a customer-centric approach. Feature teams structure has the capability to resolve many of the wastes that a component-team, waterfall process oriented, would create. The downside to a feature team is that it introduces change and challenges. 

How feature teams work.  With the Scrum methodology there is usually a Scrum Master, that facilitates the work. They would also remove impediments and distraction to help the teams deliver on time and even speed up the releases.  Different feature teams can work on a same big feature – every team has a different smaller task, working at the same time with shared code. The Scrum Masters and product owners split that work between team specialties by demand. If the work changes, the features a team works on my flex to adapt. 

Can feature teams work remotely together. During the Coronavirus the entire world figured out ways to work effectively together. The question is no longer about working remotely, but about the best ways to be effective working remotely. Managing the transition with an all-remote team could be a challenge.

The Scrum Master role. For teams early in the transition, the Scrum master likely runs the ceremonies of standup, retrospective, and sprint review. The Scrum Master guides the flow, looks out for bottlenecks, and can work as first-line resolve of issues. For example, if the team committed to pair programming at the end of the retrospective for the next sprint, and the Scrum Master sees the person is not keeping their working agreement, the Scrum Master can involve themselves. The Scrum Master can also help resolve impediments outside of the team. High functioning teams typical see the Scrum Master as enabler and teacher; the role often goes away in nine to eighteen months.

Documentation. To paraphrase Kent Beck, documentation needs to be maintained or else it goes out of date. Thus, it slows you down. The few documents Agile teams keep tend to be light and valuable - code, tests, and stories. When external documentation is required, the team may have a documentation role, who is a member of the technical staff, or may rotate the responsibility..

Fundamental tools for agile development

Short iteration and development cycle limit debugging problems, as the number of things that could have gone wrong in the last day is limited to the number of changes in the last day. With multi-month "phases", debugging and fixing become an entirely different issue.

Combining short sprints and automated regression checks for every new build is the key to continuous development and continuous integration. It speeds up the whole testing process because the changeset is so much smaller. 

If you want to learn more about the benefits of test automation you can read this article

Agile development and mobile apps

Mobile application development may be an ideal environment to implement Agile processes: apps have a short release cycle, with bugs and features removed and added quickly so that end users do not lose interest in the app. Teams doing Mobile development tend to "do" something very much like Agile, weather they call it that or not. As we've described it, that makes the essence of Agility just the only way to be successful. That keeps iterations and releases short, which makes clients and user happy, while providing for the ability to react fast to user’s requirements while delivering a high-quality product.

There is much more to current leading Agile Software Development approaches, including continuous monitoring of production, reducing mean time to recovery, getting customer feedback early, defining requirements as user stories and creating executable examples. For today, this article promised to describe how to get started.

Break the work in something one team can actually do, scope the work to something they can get done in a sprint, collaborate radically, and figure out how to scale the testing. Teams that can do that are off to a great start.

Written by 
Ely Hechte

Revised by
Matthew Heusser, August 2020

Written by

Matthew Heusser