Archive for the ‘Becoming Agile’ Category.

Software engineering isn’t

There’s a long tradition (since 1968, according to Wikipedia) of looking at software development as an engineering discipline. But calling software development engineering is something like adhering to the letter of the law, rather than to the spirit

Software development is about acquiring knowledge

If you think of software development as strictly an engineering discipline, you might be inclined to believe that the job of a software engineer is to apply his or her previous knowledge and training toward solving a particular problem. And you’d be right, to a point, because knowledge and training are important aspects of software development.

But you’d also be missing the key factor, that software development is mostly about acquiring new knowledge. Software developers are always engaged in the following pursuits:

  • Discovering what to build
  • Discovering how to build it

This goes a long way toward explaining why so much software is custom software. If software development were entirely an engineering discipline, it is conceivable that we would be able to construct just about any software application by plugging together the proper parts, chosen from a catalog of well-known, existing components. The fact that we aren’t even close to being able to do that is an indication that we still have a lot of discovery ahead.

Discovering what to build

How do we discover what to build? By working closely (and continually, if possible) with the customer. By getting feedback as often as possible from real users, and by constantly applying that feedback to the product under development.

Discovering how to build it

From the developer’s perspective, this is a career-long pursuit, as new techniques for building software are always appearing. As software developers, we can provide the greatest benefit to our clients by being aware of new developments in the field, but tempering that with the client’s needs to meet specific schedules and cost targets.

For any particular product, we can practice Test Driven Development (TDD) and encourage emergent design, so that we avoid imposing our own pre-conceived (engineering) notions on the product, and instead develop just what the client needs (and no more).

The 3 kinds of code

You can look at any chunk of code as belonging to one of three categories. Confusion between them plus a misunderstanding about the relative cost lead many people into trouble. With a little communication and some discipline, you can save yourself a lot of pain.

The 3 kinds

A few weeks back, I had to help smooth over a conflict between two co-workers. One was a seasoned developer. The other was a businessperson who just wanted some things added to their company’s web site. The conflict arose because they had very different ideas of the costs of software.

For their purposes, there were three plausible paths to take, matching the three kinds of code:

  • Temporary – Code written with a plan to throw away in a day, a week, or maybe a month. If it breaks, that’s ok — we were going to get rid of it anyhow.
  • Sustainable – Code that’s built to last. Well factored, well understood by the team, with strong automated tests, and generally meeting the team’s standards for something they’d happily maintain forever.
  • Half-assed – Everything in the middle. Code written by amateurs. Quick fixes that become permanent. Rush jobs. Things written when people say “we’ll clean it up later” and don’t.

When the businessperson expressed their desires, they wanted something cheap and quick. However, they didn’t want it to be fragile or throwaway; it had to be reliable and last indefinitely. They proposed bolting on an off-the-shelf solution that fell in the “half-assed” category. When the developer reacted poorly, they couldn’t understand why.

Follow the money

The misunderstanding comes because they had different perceptions of the costs:

businessperson
view
developer
view
Temporary low low
Sustainable high medium
Half-assed medium high

Given their perceptions, it’s natural that the businessperson would prefer the half-assed option. It’s more valuable to them than the temporary option. But doing things sustainably doesn’t get them any apparent increase in value, and they think it costs them more. Just as naturally, the developer resists what they see as the high-cost option.

In truth, they’re both right. They’re just looking at different sorts of costs. We can rewrite the column headers this way:

short-term
costs
long-term
costs
Temporary low low
Sustainable high medium
Half-assed medium high

The experienced developer is more aware of the long-term costs not only because they have more experience, but because long-term costs are generally thought of as their problem. As the maintenance costs of half-assed code add up, they won’t be attributed to last year’s excess schedule pressure from business stakeholders. Instead, the developers will be perceived as increasingly unproductive or unreliable.

Of course, that’s not just bad for the developers. It’s bad for the company as a whole. If a company’s costs for software continuously increase, that eventually becomes a powerful advantage for competitors who are more thoughtful and disciplined about their approach to software development.

Don’t fall in the trap

This is why many parts of the Agile world either explicitly or implicitly rule out the option of writing half-assed code.

Extreme Programming, for example, started out with a number of sustainability-focused practices, including pair programming, test-driven development, and merciless refactoring. As the book Planning Extreme Programming says, “With most of our planning, we assume that quality [is] fixed.” This continues up to the present day with efforts like Bob Martin’s Manifesto for Software Craftsmanship.

No matter what process you use, you can avoid the trap of half-assed code:

  1. Make sure everybody agrees that they are in it for the long haul.
  2. Educate stakeholders on how expensive supposedly cheap solutions are in the long term.
  3. Whenever people agree on a temporary solution, make sure that it is really temporary. In particular:
    • Agree in advance on the end date and expected issues. Write that down.
    • Budget for the costs of removing the temporary solution.
    • If a permanent solution is to follow the temporary solution, budget for that, too.
    • When the end date approaches, people will try to fudge it. Politely refuse, reminding everyone of the earlier agreement.

In addition, be constantly vigilant about half-assed code returning in disguise. For example, a lot of people get sold on outsourcing or offshoring because it looks cheap and easy. Sometimes it might be, but most of the projects like that I’ve seen are really half-assed code that got sold as sustainable. Just as dangerous is the cowboy coder who rides in to save the day, but leaves a lot of half-assed code behind for other people to deal with.

A sample weekly schedule

One of the things I get asked a lot is which meetings a team should have when. The right answer is that it varies for every team, and it changes over time. Still, it can be handy for new teams to have a starting point. In that spirit, here’s a schedule that I’ve assembled based on a few different smooth-running teams:

The schedule

When What Who
Monday, 9-10a Iteration planning & kickoff All team members
Tuesday-Friday, 9:30-9:40a Stand-up meeting All team members
Tuesday, 2-4p Product stakeholder meeting Product managers, external stakeholders
Wednesday, 10a-12p Product planning Product managers
Wednesday 4-5:30p Estimation All team members
Friday, 4-4:30p Product demo All team members, external stakeholders
Friday 4:30p-5:30p Process retrospective All team members

Isn’t that a lot?

For everybody except the product managers, that’s under 5 hours of formal meetings per week. Once a team is running smoothly, I see many people doing it faster than that.

Still, it’s much better to allocate extra time for these meetings, especially at the beginning. Making an agile transition is hard enough without the stress of blowing out time boxes. Instead of squeezing the schedule down, develop the habit of looking to end meetings early. Only once you consistently end early should you shorten the schedule.

Even as scheduled, this doesn’t end up feeling like a lot. The daily stand-up should be fast-paced and energizing. The meetings that developers must attend are all at the beginning or end of the day, yielding large blocks of uninterrupted coding time. And the Monday morning and Friday afternoon meetings come at times when people are normally spinning up and winding down anyhow.

Notes and caveats

For this schedule to make sense, the team must:

  • be doing weekly iterations
  • have an on-site product manager (aka XP Customer, aka Scrum Product Owner)
  • sit all in one room
  • use lightweight artifacts (e.g., index cards, not requirements phone books)

You can divide the meetings up into three groups:

  • The iteration planning and daily stand-ups cover what we are doing now.
  • The product stakeholder, product planning, and estimation meetings figure out what we may be doing soon.
  • The product demo and process retrospective look at what we just did.

The notion is that although the team is mainly focused on what’s going on now, product managers (and sometimes designers) need to work a bit ahead, so that the team can move smoothly into the next iteration.

It’s also important to note that plenty of conversation happens outside of these times. That’s why we all sit together. Rather than trying to make a schedule that formalizes every bit of useful discussion, it’s better to schedule only the minimum number of meetings to get everybody in sync.

Feedback wanted

Does this leave you with questions or suggestions? Let me know, and I’ll cover them in a future post!

Short into long: long-term planning in agile processes

Upon hearing about the short cycles of agile processes, some people are afraid that long-term planning never happens. They fear that teams could paint themselves into corners. That’s a risk: some teams do get fixated on each upcoming delivery, never sparing a thought for the future. But that’s not necessary, and below I tell you how to avoid it.

The secret

One of the secrets of agile processes is that they don’t tell you precisely what to do. Agile methods create contexts where people learn to do the right thing naturally.

For example, leaving loose ends is bad. You could make rules about exactly what constitutes a loose end and how to get rid of them. Many try.

Instead, you could start each week with a new set of work that you expect to fill the week. Then you release at the end of every week. That leaves no time for last week’s loose ends, which will turn up again as this week’s urgent bug fixes. In that context, teams have a strong incentive to learn how to tidy up all the loose ends that matter.

The secret, applied

So how do we create a context where your team will naturally spend the right amount of time doing long-term planning?

Step 1: Create a strong need for regular short-term plans. The best way to take care of this is through regular, frequent releases. If you are releasing something every week, then you are never more than 7 days from needing to have something to ship.

Step 2: Make a home for the long-term plan. There are always more good ideas than time to do them in. Instead of just throwing away the excess, you should keep them in a backlog. When you get enough of them that you need to organize them, make sure you do: that’s how the plan’s structure will emerge. For maximum effect, your backlog should be publicly visible, easy to update, and visually interesting. I usually use index cards and a rack or board. Do what works best for you, but if your plan becomes stale or gets ignored, try something more obvious and easier for everybody change.

Step 3: Establish feedback loops. It isn’t enough to release. You have to pay attention to what happens when you release. The right way depends on your environment. But observing what happens is the only way you can separate your good notions from your bad ones. The more often you do that, the better you’ll get.

Step 4: Frequently go over the plan. People with varied backgrounds should often walk through the plan together. Regular meetings are one way. But every question that starts with “why” is an opportunity to take a fresh look. As is the arrival of new data from customer feedback, user testing, market analysis, or economic forecasts.

Step 5: Be accountable to each other. A good team is in it to win, and win together. Mutual accountability helps a lot with that. Developers, for example, are responsible for building things that work for the long term, and the whole team should hold them to account for that. Equally, those managing the product are accountable to the team for the decisions they make.

Step 6: Be accountable externally. Almost every team has executives or investors that they are responsible to. On a regular basis (e.g., quarterly) tell them what you will do, what you have done, and how that compares to what you said you’d do. If they don’t ask good questions, get better advisors, even informal ones. As Doug Carlston, founder of Broderbund, has said, “The number one reason for bad software is too much money.” A lack of accountability is a big part of that.

Step 7: Keep time free for idle thought. There are many reasons to avoid being habitually overcommitted. But I think the biggest one is that tired or panicky people underinvest in the long term. We’ve all seen people come back from vacation filled with thoughtful observations and new ideas. Having good ideas once a year isn’t enough!

Some readers may feel like I’ve cheated them a bit. I haven’t told you how to break a project down into features. Or how to calculate value for units of work. There’s nothing about how to prioritize, estimate, organize, evaluate, or execute features. There’s nothing on market penetration or customer satisfaction or pleasing early adopters. Nothing on revenue models or organizational politics.

But that’s ok. I trust that given thoughtful practice, your team will figure all that out. The important thing is to start your practice right away.

21 ways to hate pair programming

Every time I’ve had the chance to talk in detail with people who hate pair programming (where two developers work jointly on the same task, usually sharing one computer, screen, and keyboard), I discovered that they were doing it in a way I too would have hated.

Below is a list of mistakes I’ve heard or seen real people make, including many that I’ve made myself. If you think pairing is not for you, first check to see if you and your team have solved all of these problems.

How to pair badly

    1. Hog the keyboard – You’re obviously superior to your partner. Control the action at all times.
    2. Ignore your partner – That noise that sounds kinda like useful suggestions? Probably a mosquito.
    3. Ignore yourself – Speaking up for your own interests can’t possibly be helpful. Suppress, suppress, suppress!
    4. Sit where you can’t see – You’ll be able to read the code when it’s your turn. If you still care then.
    5. Sit where you can’t reach – Why be ready to take action? There’s probably not much you can do anyhow.
    6. Take a back seat – Don’t sit next to your partner. Sit behind her. Then she doesn’t know when your eyes are closed.
    7. Use a regular desk - Desks set up for one person work even better for two. Plus, the furniture police might catch you!
    8. Don’t explain – Your partner, who is psychic, can just read your mind. Just keep on typing!
    9. Don’t listen – You already know what your partner is thinking. And it’s boring.
    10. Don’t ask – No, there isn’t a better way to do it, and your partner never forgets anything. So hush.
    11. Interrupt frequently – ZOMG! They made a typo! You’d better tell them right now. They’d never notice otherwise.
    12. Get distracted – Look, new mail! Hey, an IM! And check out Slashdot! Wait, there’s coding going on?
    13. Daydream - Since you’re not really there to help, it’s ok to figure out exactly how many light bulbs there are in the room.
    14. Be a back-seat driver – Your partner always wanted to be a stenographer. Keep telling them just what to type.
    15. Don’t take breaks – Everything is better when you’re tired and cranky. Especially teamwork!
    16. Work too much – It’s not what you create. It’s how many hours you spend at your desk.
    17. Ignore ergonomics – The hunchback look is back in fashion. And who doesn’t like pain?
    18. Betray trust – During, taunt about minor knowledge gaps. After, tell everybody how dumb your partner is.
    19. Refuse to learn other tools – Emacs is obviously superior to whatever junk your team uses. To pair with you, they should all learn it. Now.
    20. Don’t rotate pairing partners - If pairing for two hours is good, then six weeks with the same person must be way better!
    21. Be arrogant – Seriously, what could you learn from that other guy? You know already: nothing.

      Which ones are your (un)favorites? And what ones are missing from this list?

      Doing too much: worse than too little

      The last couple of years, I’ve run a big San Francisco race, Bay to Breakers, and it’s coming up again in May. I’m not much of a runner, and I’ve slacked off lately. But if I’m going to survive 12k with hills, now’s the time to start training.

      How doing too much hurt me

      The first part of my training is pretty simple. For 30 days, every day I do 30 minutes of mixed walking and running. (Want to start yourself? Use Hal Higdon’s plan.) But two days ago, I did too much. I met up with a more fit friend, and I let her set the pace. I did a lot more running than usual, and because it was fun to run with a friend, I did it a lot longer than usual: 60 minutes. I felt great, but like I had run a race, not gotten a workout.

      It was fun, but I paid a price. The rest of the day I was a little loopy, a little unproductive. And then the next day, I was sore and tired enough that I didn’t work out at all. I knew I was supposed to go, but I just couldn’t bring myself to get out the door. And heck, maybe I should have taken a day off. Maybe my body needed the break. Doing extra seemed like it would be better, but it caused me to break training, and in the end, I got no more running in.

      How it hurts teams

      All that reminds me of a common novice mistake: overestimates causing productivity crashes.

      Imagine a team just getting started with an Agile method like Extreme Programming. The first week, they guess they can do 20 points. However, they only get 10 done. How many should they pick to do the second week?

      • 30, because they’re behind and want to catch up;
      • 20, because the first week was an anomaly, and now they’re sure they can do 20;
      • 10, because the evidence says that’s what they can do in a week; or
      • 7, because they bit off too much and want to recover.

      The most common answers are 30 and 20. They are wrong.

      When people say that they are “behind”, that’s an indicator to me of a non-Agile culture. Their initial guess was just a guess. It is always a mistake to turn guesses into promises or plans, no matter how much stakeholders want promises or plans. A team picking 30, trying to catch up, is prone to spiraling out of control. They will repeatedly up the ante to gain back lost credibility, and repeatedly fail, either by not meeting goals or by producing junk.

      Trying again to meet an unrealistic goal, as in 20, is a more insidious mistake. Especially early in an Agile adoption, a team is trying to learn how to do things right. But if they are continuously over-committing, they won’t have time to learn. Failing repeatedly to meet goals will discourage them. Regular over-commitment prevents successful Agile adoption.

      The right answer

      In the example above, I’d be happy if the team picked either 10 or 7. Why?

      Guessing that they’d do next week what they did last week, picking 10, is an agile practice known as Yesterday’s Weather. It’s a realistic, data-driven answer, and assumes that you can get done about what you got done. There’s a risk that the team has still bitten off too much, in which case they’ll miss the goal again, but Yesterday’s Weather will sort that out in an iteration or two.

      But what if they pick too little and say 7? Well, too little may not be too little. After blowing an iteration, the team can use a little slack to recover, to figure out what the problem is, and to fix things. But if it really is too little? That’s fine. It sets a team up to exceed the goal by pulling in something extra. That makes everybody happy, and it’s a habit you want your teams to develop.

      Until a team has a strong record of regular quality output and consistently meeting goals, always err on the side of biting off too little. Challenging your limits is great, but first be sure you know what those limits are and how to recover from pushing them. When making plans or promises, do it based on a demonstrated track record, not on what you or your employers would like to be true.

      If it’s important, never stop!

      One of the most common novice questions about Agile methods is, “When do we do X?”

      People always ask this about something important, and I’ve heard it asked about a bunch of things, including design, research, testing, architecture, and optimization. I think that’s a very reasonable question, as other methods have phases for things like that. From the novice perspective, we’ve taken away the phases, so it sure can look like we’ve just discarded everything but the coding.

      The agile approach

      My answer is that if something is important, you should never stop doing it. Make sure a little of it happens every week, and maybe every day. Close those plan-act-evaluate loops as often as possible:

      • Is the user experience important to you? Then have a designer in the room, giving continuous feedback to developers. Do regular user testing. Instrument your production software and release it often, so you can see how well your theories play out.
      • Is reliability important to you? Then involve QA as part of the definition of the story. Build your acceptance tests alongside the production code. Write your code test first.
      • Is maintainability important to you? Then never stop improving the design of the code through refactoring, promiscuous pairing, and collective ownership of the code.
      • Is making the right product important to you? If so, always have a product manager in the room. Make sure people talk about the why of a feature, not just the what.
      • Is productivity important to you? Then have the team look back every week and find some way to improve your process. Encourage everybody to always be on the lookout for ways to do things better.

      And so on, for everything that you think is important to your project.

      The error of phases

      A big mistake of phased processes is to think that you can get a great product by ignoring important activities for weeks or months at a time. But people only stay good at things that they do frequently. Even worse, spending months with your back turned on some important facet of your project inevitably harms it. As humans, it’s inescapable that we forget the past and insufficiently understand the future. And new information is always coming in, no matter what phase you’re in.

      In truth, creating a cohesive product for an ever-evolving environment and audience requires regular attention to every aspect that matters. Instead of treating some important perspective as in the past or part of the future, bring them all into the present. If some activity really matters to your project, make sure somebody on your team is thinking about it every day.

      Clicky Web Analytics