One of the reasons traditional processes fail is that they are fitted to how people think they think, not how they actually do. Below I link to two interesting articles about how people think, and talk about the software process lesson I draw from them.
Waiting for marshmallows
Ninja programmer Brian Slesinsky recently pointed out to me this great New Yorker article: “Don’t: the secret of self-control.” 40 years ago, a psychologist performed a simple experiment on small children: He told them they could have one marshmallow now, or two if they waited a bit. Unsurprisingly, some children managed to do it; some didn’t.
The interesting part is why, and what happened later. Following up over the decades, it turns out this sort of self-control is powerfully predictive of success in life. One researcher found that the ability to delay gratification was better than IQ scores for predicting grades.
But how did this work? Do some kids just have more willpower than others? Was it just that some didn’t have all the facts about what they’d get when? No, it turns out. Instead, the children best at waiting put the marshmallow out of their minds. They covered their eyes, or sang Sesame Street songs. Focus on the marshmallow and you were guaranteed to lose. In other words, the successful children changed their situations to support the right sort of thinking.
Waiting for foreclosure
In this weekend’s New York Times there was an article “My Personal Credit Crisis”. The writer, Edmund L. Andrews, opens:
If there was anybody who should have avoided the mortgage catastrophe, it was I. As an economics reporter for The New York Times, I have been the paper’s chief eyes and ears on the Federal Reserve for the past six years. [...] I wrote several early-warning articles in 2004 about the spike in go-go mortgages. Before that, I had a hand in covering the Asian financial crisis of 1997, the Russia meltdown in 1998 and the dot-com collapse in 2000. I know a lot about the curveballs that the economy can throw at us. But in 2004, I joined millions of otherwise-sane Americans in what we now know was a catastrophic binge on overpriced real estate and reckless mortgages.
Again, knowing the facts weren’t enough. There were few people on the planet with more information about the topic, and he had unfettered access to people who knew the rest. But when he put himself in a situation where circumstances combined to reinforce his desires, mere knowledge wasn’t enough to save him.
So what’s this have to do with Agile development? I see two related lessons:
Think, for a moment, about the classic waterfall dream:
- you work hard to make a big plan,
- you follow the plan for a long time,
- you produce what you planned,
- and then you have a big success.
This can go off the rails in a number of ways, but key to many of the failure modes is that it mostly avoids contact with the external world up until the end. People spend most of their time in a context where their focus is on conforming to other people’s notions, rather than the real environment.
On a truly Agile project, you try to avert this through particular practices, like frequent releases, weekly iterations, and energetic engagement with users and stakeholders. Rather than pretending that you can imagine your way to a good product, you put yourself in a situation where the environment pushes you toward a great product.
This has parallels on the technical side.
All good programmers like to build things. In the wrong context, this can lead to developer gold-plating and infinite infrastructure. I once saw (but, thank goodness, didn’t work on) a system that spent $120 million on years of planning, development, and initial rollout. Many years into the project, it only had 4 working screens, which wasn’t enough to be useful to anybody. But it had plenty of infrastructure.
More insidiously, technical choices tend to be self-reinforcing. Suppose in the first week of a project a team starts out with an SQL database, just because it’s convenient. If you point out to the programmers that they’re making a big technical choice on the basis of minimal data, they will say: of course we can change the design later. But there’s a reason that those programmers work for a paycheck while Oracle’s Larry Ellison has $20 billion in the bank and collects exotic jets. Only Larry understands that very shortly they will have a hard time even conceiving of making that change.
To avoid these perils, it’s important to follow practices like simple design, pair programming, automated testing, merciless refactoring, and YAGNI. But just as important is to think just like great product managers do: treat the project as a series of small bets. Instead of grand architecture visions, it’s more effective to continuously run small experiments that help you decide between different design options.
If you structure your environment to force continuous learning and small, feedback-driven steps, your thinking will come to reflect that. And you’ll be better off for it.