Extreme Programming (XP), Scrum and Kanban emerged from the rescue of so called “Death March Projects” – projects that had been failing to deliver results, and were already very delayed and over budget. XP emerged mainly from the Chrysler Comprehensive Compensation project, one of the first projects that the Scrum community points to is the FBI Sentinel project, and Kanban points to troubled projects in Microsoft. In all cases, a defined architecture and clear product vision already existed, and it was simply a matter of removing the impediments to getting features out the door.

As development teams tried to apply these Agile methodologies to new projects, many, including O&B, encountered a number of challenges.

Since Agile has disdain for “Big Upfront Design”, many of us went to the opposite extreme, jumping into full-blown iterations with a full team, hoping that the architecture would just somehow emerge along the way. This led to a lot of painful architectural issues that were hard to fix in the middle of the project, or worse, when something was already in production. Schema changes to production systems were especially painful, as they would often entail prolonged downtime and the risk of data loss. Or what if we chose the wrong programming language or framework? Do we even have the right development team?

It was even worse when we found out that the business side was also unprepared, without a full understanding of user needs and without a well-formed product vision.

Validated Architecture

I’m not advocating for a return to Big Upfront Design. What I espouse is start with trying to achieve a “Walking Skeleton” of an architecture. Not a just a bunch of drawings, but just enough code – Spikes – to validate that the chosen approach is good enough to hang needed features.

One Sprint isn’t enough to do this. For example, working out the key entities and acceptance tests needs repeated iterations and collaboration with the Product Owner and other stakeholders. If performance is a concern, doing some load testing on some spikes will take time as well.

This sort of work should be done with just one or two senior developers and perhaps a very senior BA or domain expert, working closely with the stakeholders. There’s no point in putting together a full team at this point. We don’t even know what skills the team needs until a validated architecture is in place. Do we need Python devs or C# devs?

Validated Customer Needs

It’s even worse if we find out that there are still gaps in the Customer Discover process, and product vision. The most expensive mistake is building features that customers don’t need.

If Customer Discovery or product vision still has gaps, the Product Owner needs to go through a Lean Startup process before involving any developers. This process takes at least several months.

Respect the Product Lifecycle

The full product life cycle starts from Customer Discovery, then validating an appropriate architecture, before investing in full-blown iterations. Rushing either phase leads to expensive mistakes that are tough to fix later on, and many products just end up carrying such technical debt that’s too expensive or risky to remove.