Archive for May 2009

Marshmallows, mortgages, and metacognition

One of the reasons traditional processes fail is that they are fitted to how people think they think, not how they actually do. Below I link to two interesting articles about how people think, and talk about the software process lesson I draw from them.

Waiting for marshmallows

Ninja programmer Brian Slesinsky recently pointed out to me this great New Yorker article: “Don’t: the secret of self-control.” 40 years ago, a psychologist performed a simple experiment on small children: He told them they could have one marshmallow now, or two if they waited a bit. Unsurprisingly, some children managed to do it; some didn’t.

The interesting part is why, and what happened later. Following up over the decades, it turns out this sort of self-control is powerfully predictive of success in life. One researcher found that the ability to delay gratification was better than IQ scores for predicting grades.

But how did this work? Do some kids just have more willpower than others? Was it just that some didn’t have all the facts about what they’d get when? No, it turns out. Instead, the children best at waiting put the marshmallow out of their minds. They covered their eyes, or sang Sesame Street songs. Focus on the marshmallow and you were guaranteed to lose. In other words, the successful children changed their situations to support the right sort of thinking.

Waiting for foreclosure

In this weekend’s New York Times there was an article “My Personal Credit Crisis”. The writer, Edmund L. Andrews, opens:

If there was anybody who should have avoided the mortgage catastrophe, it was I. As an economics reporter for The New York Times, I have been the paper’s chief eyes and ears on the Federal Reserve for the past six years. [...] I wrote several early-warning articles in 2004 about the spike in go-go mortgages. Before that, I had a hand in covering the Asian financial crisis of 1997, the Russia meltdown in 1998 and the dot-com collapse in 2000. I know a lot about the curveballs that the economy can throw at us. But in 2004, I joined millions of otherwise-sane Americans in what we now know was a catastrophic binge on overpriced real estate and reckless mortgages.

Again, knowing the facts weren’t enough. There were few people on the planet with more information about the topic, and he had unfettered access to people who knew the rest. But when he put himself in a situation where circumstances combined to reinforce his desires, mere knowledge wasn’t enough to save him.

So what’s this have to do with Agile development? I see two related lessons:

Product dreams

Think, for a moment, about the classic waterfall dream:

  1. you work hard to make a big plan,
  2. you follow the plan for a long time,
  3. you produce what you planned,
  4. and then you have a big success.

This can go off the rails in a number of ways, but key to many of the failure modes is that it mostly avoids contact with the external world up until the end. People spend most of their time in a context where their focus is on conforming to other people’s notions, rather than the real environment.

On a truly Agile project, you try to avert this through particular practices, like frequent releases, weekly iterations, and energetic engagement with users and stakeholders. Rather than pretending that you can imagine your way to a good product, you put yourself in a situation where the environment pushes you toward a great product.

Technical dreams

This has parallels on the technical side.

All good programmers like to build things. In the wrong context, this can lead to developer gold-plating and infinite infrastructure. I once saw (but, thank goodness, didn’t work on) a system that spent $120 million on years of planning, development, and initial rollout. Many years into the project, it only had 4 working screens, which wasn’t enough to be useful to anybody. But it had plenty of infrastructure.

More insidiously, technical choices tend to be self-reinforcing. Suppose in the first week of a project a team starts out with an SQL database, just because it’s convenient. If you point out to the programmers that they’re making a big technical choice on the basis of minimal data, they will say: of course we can change the design later. But there’s a reason that those programmers work for a paycheck while Oracle’s Larry Ellison has $20 billion in the bank and collects exotic jets. Only Larry understands that very shortly they will have a hard time even conceiving of making that change.

To avoid these perils, it’s important to follow practices like simple design, pair programming, automated testing, merciless refactoring, and YAGNI. But just as important is to think just like great product managers do: treat the project as a series of small bets. Instead of grand architecture visions, it’s more effective to continuously run small experiments that help you decide between different design options.

If you structure your environment to force continuous learning and small, feedback-driven steps, your thinking will come to reflect that. And you’ll be better off for it.

A sample weekly schedule

One of the things I get asked a lot is which meetings a team should have when. The right answer is that it varies for every team, and it changes over time. Still, it can be handy for new teams to have a starting point. In that spirit, here’s a schedule that I’ve assembled based on a few different smooth-running teams:

The schedule

When What Who
Monday, 9-10a Iteration planning & kickoff All team members
Tuesday-Friday, 9:30-9:40a Stand-up meeting All team members
Tuesday, 2-4p Product stakeholder meeting Product managers, external stakeholders
Wednesday, 10a-12p Product planning Product managers
Wednesday 4-5:30p Estimation All team members
Friday, 4-4:30p Product demo All team members, external stakeholders
Friday 4:30p-5:30p Process retrospective All team members

Isn’t that a lot?

For everybody except the product managers, that’s under 5 hours of formal meetings per week. Once a team is running smoothly, I see many people doing it faster than that.

Still, it’s much better to allocate extra time for these meetings, especially at the beginning. Making an agile transition is hard enough without the stress of blowing out time boxes. Instead of squeezing the schedule down, develop the habit of looking to end meetings early. Only once you consistently end early should you shorten the schedule.

Even as scheduled, this doesn’t end up feeling like a lot. The daily stand-up should be fast-paced and energizing. The meetings that developers must attend are all at the beginning or end of the day, yielding large blocks of uninterrupted coding time. And the Monday morning and Friday afternoon meetings come at times when people are normally spinning up and winding down anyhow.

Notes and caveats

For this schedule to make sense, the team must:

  • be doing weekly iterations
  • have an on-site product manager (aka XP Customer, aka Scrum Product Owner)
  • sit all in one room
  • use lightweight artifacts (e.g., index cards, not requirements phone books)

You can divide the meetings up into three groups:

  • The iteration planning and daily stand-ups cover what we are doing now.
  • The product stakeholder, product planning, and estimation meetings figure out what we may be doing soon.
  • The product demo and process retrospective look at what we just did.

The notion is that although the team is mainly focused on what’s going on now, product managers (and sometimes designers) need to work a bit ahead, so that the team can move smoothly into the next iteration.

It’s also important to note that plenty of conversation happens outside of these times. That’s why we all sit together. Rather than trying to make a schedule that formalizes every bit of useful discussion, it’s better to schedule only the minimum number of meetings to get everybody in sync.

Feedback wanted

Does this leave you with questions or suggestions? Let me know, and I’ll cover them in a future post!

Start at the end

Any software development effort involves the conveying of information, but it’s easy for the important information to be hidden by related (but less important) stuff. Starting at the end is a good way to focus on the information that matters most.

Drawing Maps

In the days before Google Maps, we used to ask other people for directions to places we wanted to go, and often someone would volunteer to draw a map on paper. After taking part in many of these information exchanges, I observed that most people (including myself), when drawing a map, start at the beginning. That is, they start the drawing at the traveler’s point of origin and proceed from there to the destination.

The result was almost always adequate for the intended purpose of getting the traveler from point A to point B, but I noticed that the destination often ended up being crammed into a corner of the page, almost as if it were an afterthought rather than the most important piece of information on the map.

Now it certainly makes sense to draw a map in this fashion. I imagine that the main reason for doing it this way is that when drawing a map we imagine ourselves actually taking the trip we’re describing. We start at the beginning of our imaginary trip and eventually find our way to the destination.  And unless we begin with an inordinately large piece of paper, or are particularly good at allocating space ahead of time, we end up with a map that contains extra information we really don’t need, while details of the destination are omitted because we run out of room.

So I started drawing maps from the end. The first place I put on a map is the destination. From there I work backwards, drawing the streets leading to the destination. Then, if I have room, I draw the highways leading back to the city of origin. The result is a map that emphasizes the important details.

Communicating Technical Information

I’ve noticed a similarity between the way people draw maps and the way they convey technical information. That is, they often tend to start at the beginning and proceed in chronological order to the end.

Just as in map drawing, this approach makes a certain amount of sense. It’s natural for people to convey information in the order in which they discovered or developed it. The communication of the information is kind of a chronological account of the original experience of creation.

The problem is, it’s the end (the destination) that I’m most interested in. I tend to think of the result as the important part, with everything leading up to it being more like “supporting information” which I may or may not want (or even need) to know.

In other words, I tend to get lost in the details of the journey, when what I’m really interested in is where I’m going to end up.

Writing Software

I used to write code from top to bottom. That is, when I wrote a method I would start at the top of the method, and write lines of code in the order in which they would be executed. It made sense, in a way, because I was taught to read and write English from top to bottom. It seemed the natural way to go about it. Unfortunately this approach tended to introduce unnecessary code into the method that had to be stripped out later.

Now when I write software, I like to start at the end. When I write a unit test, for example, I start by writing the assert statement, even if it asserts against objects that don’t yet exist. I start with the assertion, then I see what the assertion requires and I write a line of code above it to satisfy those requirements. If satisfying those requirements introduces new requirements, I write a line of code above that, and so on.

The result is a unit test that carries no extra baggage, no superfluous information, with an emphasis on the most important part, the assertion.

The assertion in a unit test is analogous to the destination on a map. Both are the most important pieces of information. Both make excellent starting points.

Clicky Web Analytics