Standard Kanban boards

The core notion of a software development Kanban board is that you represent visually the state of various units of work. These units of work are pretty small, generally a few person-days. A basic one looks something like this:

Stories flow in from the backlog on the left. After they are worked on (D-E) and reviewed (F), they are released and put in the file of finished stories, rarely to be looked at again. There are generally limits, so that a team will only have a certain number of stories in each stage at one time. Because the code could be around for years, we need to make sure to write it in a sustainable way, and the Agile movement, especially the Extreme Programming end of it, has discovered and promoted a lot excellent techniques for making code that lasts.

This flow makes a lot of sense when you are using Agile techniques to solve a known problem, and have pretty high confidence that the things you build will last indefinitely.

Lean Startups are different

However, in software startups, and especially when following the Lean Startup approach, you are solving an unknown problem. The goal, especially in the early stages, isn’t to build software; it’s to discover needs and wants that people will pay you to satisfy.

The standard startup approach, typified by dot-bomb Webvan, is known in the Lean Startup community as the “field of dreams” approach. You build a complete product in the hope that when you finish, your customers will come. The Lean Startup approach instead requires that you identify your hypotheses about your customers, your market, and your products, and test those hypotheses as early as possible, partly by trying a lot of things out and seeing how they work.

Another way to look at this comes from Eric Ries:

Anything that makes that loop longer is worse. Technical debt can surely do that. But if you write something to last the ages and then remove it next week, that can also slow down your feedback loop.

My answer is to explicitly accept there will be two kinds of code in the system. There will be sustainable code, the kind that Agilists have spent the last 15 years learning how to create properly. And there will be temporary code, code that we explicitly plan to throw away. I still think the third kind of code, half-assed code, has no place in a serious software project. But there’s a risk, one many of us have lived: what if the temporary code sneaks into the permanent side, putting the project on the road to a technical debt bankruptcy?

Lean Startup Kanban boards

My solution to this is to expand the Kanban board to explicitly represent the flow of experiments. Below, experiments are represented in blue:

For the early part of their lifecycle, experiments and regular features share a pretty similar path. They hang around in a backlog until the are put on deck (C). They get worked on by the team as part of the regular flow (E), and accepted by product managers (G). But then, after release instead of being put in the released-and-forgotten box, those stories are tracked as active experiments (H-M).

When the experiment is concluded, then the feature might be accepted permanently, in which case the code will need to be cleaned up, and the rest of the system refactored to accommodate the new code. More likely, the code will be removed. Possibly because the feature turned out to be a bad idea, at least in its current form.

Or possibly it was a good idea, but not good enough in its current form to keep.  Often the first approach to a feature (or to its implementation) isn’t the right one; it can be better to remove the experiment and its code in favor of an upcoming correct implementation. In which case, it will yield a number of new story cards for the backlog.

I think this approach will mitigate the biggest risk of accepting any temporary code: you forget that it’s temporary, and suddenly you’ve got technical debt in your system and no scheduled time to clean it up. There are other risks, some of which I hope to write about over the coming weeks and months.

Have thoughts on this? Have you tried something similar? I’d love to hear about it. Leave a comment!

The Agile reaction

In the dark days of waterfall, code bases had only two states: being re-written or slowly getting worse. The project would start out with great hope. People would write something beautiful, launch a 1.0, and then spend their time patching it. Inevitably, and despite initial efforts to resist, the code base would go slowly downhill. Gradually increasing technical debt would lead to technical bankruptcy, forcing a big rewrite. Then the agonizing cycle would start again.

One of the key insights of the Agile movement, one crystallized by Martin Fowler’s book Refactoring, was that decay was not inevitable. We could clean things up incrementally as we went, without needing periodic giant rewrites. Another key insight comes via Kent Beck’s Extreme Programming: if we worked iteratively and refactored all the time, then we could start clean and stay clean, forever, ending the cycle of giant rewrites.

As easy as that sounds, it’s actually very hard; every solid Agile practitioner has spent time developing a nose for technical debt, and a hate of it. So when Kent Beck suggested that startups could responsibly create some technical debt, it caused a lot of strong reactions: to some people it sounded like the pope saying that premarital sex and adultery were occasionally fine.

But startups are about learning

For this to make sense, you have to understand two things:

1. Startups—even software startups—are businesses, not software development projects.
2. The point of a startup isn’t to create a software product; it’s to discover a sustainable business model.

In a startup, software development isn’t an art for its own sake; what makes sense from a pure engineering perspective may not make sense more broadly. That’s true for any business really, but it’s especially true for a startup. That’s because the main goal of a startup is to discover a valuable business. Discovery requires experimentation, and the amount you can learn is a function of the cost of your experiments.

So suppose I have a new idea for a game. I believe it will be A) fun and B) a moneymaker. I could take a number of months and design it, develop it, polish it, and publish it. Then the market would tell me whether my beliefs were correct. Or I could make a very crude version with 1 level, have some people play it, and see if the core gameplay is enjoyable. That’s my first experiment to examine hypothesis A, fun. If it works, I could put it up on the web, buy a few ads, and see if the game sounds interesting enough to click through and try it. That would be my first test for hypothesis B, making money with the game.

Now if these tests fail, then my code base may be pretty short lived. They say that hard work pays off eventually, while laziness pays off now. The same is true for good development practices. A lot of Agile practices get you long-term sustainability, but they cost you in the short term. For sufficiently small and short-lived pieces of code, worrying about technical debt does not make business sense. Better to invest in more experiments.

Danger Will Robinson!

At this moment, some of my readers are leaping up to say “But! But! But!” That’s what I wanted to do during Kent’s talk, anyhow. Because when businesspeople hear that they can have something quicker, they will often interrupt to say, “Yes! Do that!” As I explain in “The 3 Kinds of Code“, a lot of business/engineering fights arise because engineers are more aware of the long-term costs of short-term thinking. I have seen good teams, even ones otherwise well-versed in Agile techniques, give in to that business pressure again and again, creating giant messes.

So hacking things together, and the resultant technical debt is extremely dangerous, but sometimes very powerful. What should a responsible team do?

I believe there are good ways to handle this dilemma, and will propose one approach in my next post.

1. People wearing headphones. Part of the reason to sit together is to listen to one another. If team members are wearing headphones, they can’t do that. Don’t blame them, though; figure out what noise or distraction drives people to do that and eliminate the root problem. And if they aren’t part of the team, move them elsewhere.
2. Stale artifacts on the walls. Every artifact on the wall should be there for a reason. If there are a lot of stale plans, charts, or lists on the walls and whiteboards, that’s a sign of trouble. Immediately prune the junk!
3. Workspace as information desert. Development teams turn knowledge into software products. Rather than requiring effort to find things out, a good workspace requires effort to avoid knowing what’s going on. Bare walls often indicate low collaboration or high confusion.
4. Minimal interaction. If team members sit near one another and never talk, it’s often a sign of an underlying problem. I’ve seen this caused by bad relationships, code silos, excess division of labor, too-long release cycles, excess schedule pressure, and plain old shyness.
5. Furniture as barrier. Furniture should help you work, not get in the way. Barriers are great to reduce noise and chaos at team boundaries, but true teams should be able to share space and collaborate effectively.
6. Sad or ugly spaces. You will likely spend more waking hours in this room than any other. Shouldn’t it be nice?
7. Seating by job description. Agile approaches require cross-functional teams to make whole products. If people are grouped by job description, that is at least a barrier to collaboration, and often a sign of unhelpful silos. Group people by project instead.
8. Space and furniture as status markers. In some companies, being distant from the action is a sign of status. On Agile teams, that’s a mistake. Instead of using rooms and desks to indicate hierarchy, give people the tools they need to do their jobs.
9. No laughter, no fun. This is a big one for me. Every really productive team I’ve visited enjoys the work and their fellow team members. Money can make people show up, but it’s joy that gets the best results.

That’s my list. What’s yours?

Proactively reactive

In terms of product produced, you can be more proactive in an Agile context than you can in a Waterfall context. With Waterfall methods, you just have to hope your designs and plans are correct, reacting massively after each large release. With Agile approaches you can continually test your assumptions and hypotheses, allowing you to eliminate bad ideas early and invest more resources in areas that have been proven to deliver more long-term value.

Some Agile shops may end up stuck in purely reactive cycles, with no long-term plans. (I don’t see that much.) But the good ones are continuously updating their plans based on the new information that you can only gain if you can release frequently. It has been said that Waterfall is plan-driven, while Agile methods are planning-driven. Having tried both, I think frequent plan improvement is much more proactive.

The Agile conversation

Another way to look at it is that Agile approaches proactively take advantage of people’s reactive skills by creating situations where their reactions will be maximally useful.

Conversations are in some sense essentially reactive: you say something, I respond to it, you respond in turn. But we can proactively decide to have a conversation and expect to get something useful, perhaps novel out of it.

Agile approaches have conversations with markets and user communities, using new releases to advance the discussion. We release something and say, “How about this?” People respond: they love X, they hate Y, and how did you miss Z! We release another thing and say, “Is this better for you?” And so the conversation goes, week after week.

(I was inspired to write this article by a thread in the scrumdevelopment Yahoo! group titled Executives Tracking Individual Developer Productivity?  The originator of the thread wrote about a real issue in his company that had to do with the CEO wanting to measure the productivity of the company’s developers based on the principle that  “you can’t manage what you can’t measure.”)

Productivity good!

We can probably agree that productivity (in the abstract) is a good thing, but can we agree about what it means concretely? For starters, what are the “units” of productivity for individual people or teams?

For example, we can compare the gas mileage of vehicles to each other in units of  “miles per gallon”, and we would probably agree that vehicles with higher numbers are more productive.

What units could we use to compare individuals or teams in a similar fashion? One commonly-employed notion from our past, and one that we have thankfully left behind, is that developer productivity could be measured in lines of code produced in a given period of time. But if that’s behind us, what are we left with?

Productivity defined

The business novel The Goal defines productivity as follows:

“Productivity is the act of bringing a company closer to its goal. Every action that brings a company closer to its goal is productive. Every action that does not bring a company closer to its goal is not productive.”

Sounds simple enough. Now all we have to do is to figure out what the company’s goal is, and we’re there.

Do you know what your company’s goal is? Not its mission statement, but its goal? Is your company in business for a profit, or do you work for a non-profit concern? Most of us, I believe, work for companies interested in making a profit. And if that is so, then most companies’ goals might very well be as simple as making a certain amount of profit per year.

But we’ll probably want to measure the productivity of our developers more frequently than once per annum, so we might express the productivity of an individual or team in terms of “profits earned per iteration”.

The thing is, can we actually measure the contribution to profits made by an individual developer or even a team of developers? Offhand I’d say that this is a difficult proposition, at best.

However, I contend that if we cannot correlate the productivity of an individual or team with profits, then there really isn’t much point in trying to assess that productivity in the first place.

Looking where the light is better

There’s on old joke about a man who is searching the area around a corner streetlight. A second man comes along and asks him what he’s doing, and the first man replies, “I’m looking for a quarter that I dropped.” The second man joins in the search, but neither man is able to spot the lost coin. Finally the second man thinks to ask, “are you sure you dropped it here?” The first man replies, “no, I dropped it in the alley, but the light’s better here.”

In a for-profit company, attempting to measure the productivity of a developer in any terms other than contribution to profits amounts to looking where the light is better. In other words, don’t measure something just because it’s convenient to measure.

The problem

Many real-world teams treat certain kinds of design, including visual design, user interface design, and interaction design, as quantities that you can add later. Often this is done with the best of intentions.

For example, the team’s designer may be unavailable, so the rest of the team will get to work on a story, building an obviously rough interface. If, by the time they’re done, the designer still isn’t available, the product owner might accept the story as complete, making a mental note to come back later, perhaps much later.

When somebody external comments on the ugly interface, the developers might say, “Oh, that hasn’t been designed yet.” They’re wrong.

Software is nothing but design

People often talk about software with an implied analogy to industrial production. One group of people makes up some blueprints, and then an entirely different group of people makes physical objects. That second group is judged not by the utility of the objects, but by conformance to the design. This can sometimes be a useful analogy, but in an important way it’s entirely false.

In truth, the creation of software is 100% design. The computer does all the work of making things happen in the real world. The software is just the blueprint the computer uses to decide what to do. In the same way that a good manufacturer is one that follows the instructions well, a good computer is one that executes the software faithfully and reliably. All the humans participating in the creation of software are involved in a joint design activity.

“Not designed” is badly designed

So if software is pure design, then what’s going on with the ugly interface? The notion that it is somehow “not designed” is wrong. It’s just badly designed. Why does that matter? Because good design isn’t something you can spray on later like a new coat of paint.

Programmers already know that for the kinds of design they appreciate. Good developers try hard to avoid  spewing out reams of confusing, badly organized code that they hope to clean it up later. They know that’s wasteful, and likely to hide tricky problems. Tactically, they may choose to leave certain things messy for a short period. But experienced developers do that judiciously, painfully aware of how easily a controlled low-quality situation can turn into an uncontrolled one.

Teams should have the same attitude about every kind of design that matters for their project. Each story should be well designed from every perspective before it is declared complete. That sounds like it could be a lot of work, but it needn’t be. As with the software architecture, other sorts of design can be approached incrementally and iteratively.

The only hard part is making sure that you have all key contributors working together. The easy way to do that? Put them all in a room, and have them release something every week. They’ll figure it out.

Software development is about acquiring knowledge

If you think of software development as strictly an engineering discipline, you might be inclined to believe that the job of a software engineer is to apply his or her previous knowledge and training toward solving a particular problem. And you’d be right, to a point, because knowledge and training are important aspects of software development.

But you’d also be missing the key factor, that software development is mostly about acquiring new knowledge. Software developers are always engaged in the following pursuits:

• Discovering what to build
• Discovering how to build it

This goes a long way toward explaining why so much software is custom software. If software development were entirely an engineering discipline, it is conceivable that we would be able to construct just about any software application by plugging together the proper parts, chosen from a catalog of well-known, existing components. The fact that we aren’t even close to being able to do that is an indication that we still have a lot of discovery ahead.

Discovering what to build

How do we discover what to build? By working closely (and continually, if possible) with the customer. By getting feedback as often as possible from real users, and by constantly applying that feedback to the product under development.

Discovering how to build it

From the developer’s perspective, this is a career-long pursuit, as new techniques for building software are always appearing. As software developers, we can provide the greatest benefit to our clients by being aware of new developments in the field, but tempering that with the client’s needs to meet specific schedules and cost targets.

For any particular product, we can practice Test Driven Development (TDD) and encourage emergent design, so that we avoid imposing our own pre-conceived (engineering) notions on the product, and instead develop just what the client needs (and no more).

I recently hosted a workshop that let participants experience three different ways of transferring knowlege in a production environment. The product, in this case, was a paper airplane of unusual design. The idea was to try different ways of transferring the knowledge about how to build the airplane from the “chief designer” (me) to the production workers, and to compare the relative productivity of the different methods, which were:

• Documentation - The workers were given written instructions (22 steps worth) for building the airplane.
• Reverse Engineering - The workers were given a completed airplane which they could study in order to reproduce the steps required to build it.
• Mentoring - The “chief designer” built an airplane step by step and the workers replicated each step as it was performed.

The experiment was conducted in two phases. In the first phase, all 8 participants used the Documentation method. In the second phase, one team of 4 tried Reverse Engineering, while the other team of 4 tried Mentoring.

The results were interesting. Using the Documentation method, only one person out of a total of 8 came close to being able to build the airplane at all in the 5-minute period allotted.

Using the Reverse Engineering method, 1 person out of a total of 4 produced a completed airplane in 5 minutes.

Using the Mentoring method, each of 4 team members produced a completed airplane, and in less than the 5 minutes available.

The knowledge transfer bottleneck in software

In a software development effort, knowledge transfer takes place all the time, and it’s easy to imagine a software developer in the “chief designer” role described in the exercise above.

Let’s say I’m a developer who has discovered, and written the code to implement, a technique for binding some data to the controls in a user interface, and that this technique forms a pattern that my fellow developers want to know about. If you were one of my fellow developers, would you rather I (a) gave you a document I had written about the technique, (b) told you where the code was and suggested you figure it out for yourself, or (c) paired with you to implement the pattern for a new set of data?

Now, certainly, pairing with you takes more of my time, and might seem less efficient from my viewpoint. After all, I could be off designing the next pattern, and the one after that. But the  productivity of the team as a whole, rather than my personal productivity, is what’s important. And mentoring helps increase the team’s productivity by avoiding knowledge transfer bottlenecks.

(Note: I didn’t invent the exercise described here. I first saw something like it presented at the Agile2006 conference by Ashley Johnson and Rich Phillips of Valtech Technologies, Inc.)

The assembly line analogy

I recently conducted an exercise with the folks at North Bay Agile in which two teams formed assembly lines for folding paper airplanes. Each assembly line consisted of a number of distinct operations, as follows:

• Operation 1 - Get raw material (a sheet of paper) from stock.
• Operation 2 - Fold inward lengthwise, then unfold.
• Operation 3 - Fold top corners inward.
• Operation 4 - Fold sides inward, fold in half, fold wings down.

In the first shift, every member of each team worked at top capacity. The result: Each team produced about a dozen airplanes in 5 minutes.

But then I had each team fill out a “profit and loss” statement. They got credit for “selling” all planes produced, but they were also debited for the labor and material costs of uncompleted airplanes (which stacked up in front of Operation 4, the bottleneck). The financial news: Both teams incurred a loss.

In the second shift, the teams were instructed to slow down to match the rate of the slowest operation. This was accomplished by creating a “buffer zone” before each operation and following a rule which said, “you can’t pass your work on to the next operation until that operation’s buffer zone is empty.” The result: Each team still produced around a dozen airplanes in 5 minutes.

When the profit and loss statements were filled out a second time, each team showed a profit. This was directly due to the fact that no work-in-process inventory built up, thus reducing the amount spent on materials and labor.

The most obvious difference in the overall activity of the assembly lines from shift to shift was that, in the second shift, the upstream operations were sometimes idle. By slowing the upstream operations to match the rate of the slowest operation, both assembly lines increased their productivity.

Increasing productivity stepwise

After the first shift in the exercise, the inclination of several participants was to try to find ways to improve the performance of the slowest operation. As the second shift demonstrates, however, a simpler first step is to just slow down all of the upstream operations to match the rate of the slowest operation.

The business novel The Goal outlines a process for increasing the productivity of a manufacturing system:

• Step 1 - Identify the system’s bottlenecks.
• Step 2 - Decide how to exploit the bottlenecks (e.g. don’t let a bottleneck be idle).
• Step 3 - Subordinate everything else to the above decision (e.g. throttle back the upstream operations).
• Step 4 - Elevate the system’s bottlenecks (e.g. speed up a slow operation).
• Step 5 - If, in a previous step, a bottleneck has been broken (i.e. a bottleneck is no longer a bottleneck) go back to Step 1.

 As you can see, the recommendation is to slow down all non-bottleneck operations before trying to speed the bottlenecks up.

How does the assembly line exercise relate to software development?

Although software development is not the same as manufacturing, there are situations in development that exhibit the characteristics of an assembly line. Suppose, for example, that you are a developer building components for use by other developers. If you (the upstream operation) produce components at a rate faster than the other developers (the downstream operations) can understand and use them, you can create a bottleneck, causing excess “inventory” to build up.

Software development is about creating and sharing knowledge

Here are some simple things to remember when considering whether it is more profitable to work at top capacity or to be idle part of the time:

• Knowledge is the inventory of software development
• People consume knowledge at their own rate
• Creating knowledge faster than it can be consumed causes excess inventory
• Excess inventory reduces profits

In other words, working at your own top capacity may not be the positive thing you think it is. If you’re producing software components at a rate greater than the rate at which they can be put to use, you could be hurting the bottom line.

Agile is the new black

A friend of mine is a founder at a startup that’s very Agile: weekly iterations, releasing 2-10 times a month, heavy test coverage, and lots of pair programming. He met for coffee with a friend of a friend, an executive in charge of building his company’s first significant piece of software. Part of the conversation went like this:

My friend: So, how are you developing your software?

Executive: We’re doing Agile!

My friend: That’s great! We started out two years ago with a typical Extreme Programming set of practices, but since then we’ve made a number of significant changes, including [list of new and modified practices]. And we’re still continuously improving. How about you?

Executive: We’re doing Agile!

My friend: [Sighs, shakes his head sadly.] Nice weather we’re having, isn’t it?

In my experience, this is becoming more and more common: somebody hears that Agile is what all the cool kids are doing, so they take a very shallow understanding of what that means and run with it. This rarely works out well, and always puts me in mind of somebody who goes to a buffet restaurant and makes a meal out of the desserts. It’s theoretically appealing, but the long-term costs are fearsome.

Fad-proofing your Agile adoption

Agile methods have become popular because there’s real value to them. However, just jumping on the bandwagon won’t do you any good. Take the time to think through what you’re trying to achieve. Do an Agile adoption poorly and people may become cynical enough that you won’t have another chance. Here are 10 things you can do to help:

1. Focus on what works - Your team isn’t there to create process or make software. You’re there to deliver sustainable long-term value via software. Use that as your primary ruler to measure success.
2. Don’t expect miracles - There are no silver bullets! Agile methods don’t fix your problems for you; they mainly make the problems obvious quickly, so you can solve them yourselves.
3. Give yourself room - Thinking that “doing Agile” must be quick and easy, many teams don’t allow enough time to learn how to make the practices work well for them. Allow for an initial productivity hit, one that will get paid back richly down the road.
4. Don’t get suckered - Now that “Agile” is a big buzzword, there’s an equally big incentive for consultants and authors to sell you Agile lite, the software process equivalent of cotton candy. Don’t fall for it!
5. Don’t just “do Agile” - I’ll tell you a secret: there is no “Agile” that you can do. There is a collection of Agile methods; you can do any one of them. There are a whole host of practices to adopt. But unless you know enough to tell them apart, you don’t know enough to do any of them well.
6. Get help from people who have done it - As a coach, I’m probably a little biased, but I think everybody who tries an Agile approach should involve somebody who’s done it before. Whether that’s a key player, a manager, or a coach doesn’t matter much. But it’s hard to learn Agile methods from books, let alone a couple of articles on the web.
7. Track and pay down your debt - Part of an Agile adoption is raised standards. That leaves most people with a lot of cleanup to do. Make a list of all your code debt and testing debt, and have a plan for paying it off.
8. Push power down - Agile methods do best in a context where teams are given broad latitude to solve problems on their own. Focus on supporting rather than directing your Agile teams.
9. Improve continuously - An Agile method isn’t something like a turbocharger that you bolt on once and then use forever. If you aren’t having regular retrospectives and continually improving your process, you’re not Agile.
10. Use your ignorance for good - One of the biggest mistakes people make is thinking that even though they’ve never used an Agile approach, they know enough to make big changes to it from day 1. Instead, accept that you’re trying something truly new to you, and act like it.

Feedback wanted

Have any war stories about the dangers of being fashionably Agile? How about tips on doing it right? I’d love your comments, and other readers would, too.

P.S. Hat tip to my old pal Christopher DeJong for the phrase “Agile is the new black”.