The Thumbtack and Hole Metrics...

Sometimes metrics can just appear in front of you very simplistically without any effort. I've discovered two new ones which I call the Thumbtack Metric and the Hole Metric.

The back story is that we use a corkboard to track our work. This is a high level view of the biggest priorities in VersionOne, our legacy ticket system, and any other queues of work we have. Because our one team supports all company operations AND does new product development, we have a hybrid balance of many different approaches. The board is our central view.

I attempted to force WIP limits onto the board so that it was a true Kanban board, but this simply didn't work in our environment for multiple reasons (nature of our work along with company culture). BUT, it has been very interesting for the team to start realizing when they are attempting to do too much. As the coach, this was something I used to pro-actively point out, but I'm learning that it is better for me to be reactive so that the team themselves can learn from experience. (Unfortunately, a child understands "hot" much better after being burned.)

This maturity and growth has led to many great conversations on the team. They start as individual discussions of philosophy between me and someone on the team, and then they become team discussions where we improve something in our process.

Through all of this, I've noticed a few things that the team now sees after it has been pointed out:

1. A problem item typically has many thumbtack holes in it (The Hole Metric). The "holier" the item, the more of a problem it is. What is the causation behind this correlation? Well, every week we scrub the board. Every few days rows get re-aligned. If an item is picked up on one day and finished the next... it has one or two holes in it. But if the item gets pushed around for days and lost under other stickies... then it get punched A LOT! So... we now have a metric that can be used to identify areas of improvement. When a "holy" item reaches DONE, it is a good topic for a quick review/mini-retrospective.
2. Every item on the board needs a thumbtack. We thumbtack stickies so that they don't get blown off while moving the board around (our board sits in a central visible area of the company and is carried into our dedicated standup meeting room every morning). Sometimes we group things with one thumbtack, but in general, there is a correlation between the amount of work on the board and the number of thumbtacks used. I haven't been successful at imposing WIP, but I have been successful in showing that when we run out of thumbtacks, we are in for trouble. This is the Thumbtack Metric. If you need more thumbtacks to stick things onto the board, you have a problem coming (or are already knee deep in it). Conversely, the CEO notices that too many unused thumbtacks might be a sign of available capacity (so I take thumbtacks off the board at times if needed ).
   

See... all those documents, white papers, conference sessions, etc you've read about metrics... they were overkill. There are simple metrics right in front of your team every day. AND, they are much easier to sell, explain, and use as an indicator moving forward.

So... what's your thumbtack metric?

Burndown charts as an indicator...

Jack Milunsky posted an interesting point in the Agile Project Managers group on LinkedIn. I normally agree with him and like his work (actually, I still do here)...

His core point is quite valid. The point of the burndown shouldn't be to drive a team towards focusing on the team's ability to match a perfect trend line. This is something I totally agree with. I've seen teams with perfectly straight lines on their burndown and I've learned that this is a big flag! That being said, his following sentence:

I believe that the underlying goal should not necessarily be about staying below the estimated time line in a burndown chart (although this would be nice).

..might set the wrong tone for a newbie to agile. I wanted to address that here.

Joe Blotner captures this well with the following statement:

Scrum doesn't solve your problems, just make them visible, so you can solve them.

...which I'm sure that Jack would agree with.

But... I want to state that I think the main goal of a burndown IS to trend downwards. This isn't the same as staying below the estimated line, but I want to be clear... if the trend isn't down, then you are going backwards. As a business person (PO), I would not accept a burndown going up or flat too long. This is a sign that I'm spending money and there's no sign of progress (both in uncovering all the unknowns and in making progress). I'd almost prefer seeing the line spike up and then start working back down so I know the problem is behind us. A flatline always triggers me to ask "but when will it end?"

That being said, if this is what was happening before agile, then Joe is correct. Agile has helped you focus on this and now you can try and fix it.

Little nuances I know... but I wouldn't want to see a new scrum team looking at their burndown and saying... "eh, that's okay for now... let's figure it out in a few days (or at the end of the sprint)."

Quote Series Part 4 - Metrics & Managers

This is part four of the series, to see what this series is about, click here.

Quotes on "Metrics"

Another purpose of measuring capacity is to improve throughput. If you plan for less than your capacity, you get less done than you could have. If you plan for more than your capacity, you get less done than you could have.

Having a plan with "enough" work in it less some slack to improve the probability of meeting commitments increases the amount of work that gets done compared to planning for too much or too little. - Kent Beck

Once you start measuring something, you can easily end up in a situation where the measurement itself starts influencing the things you want to measure. - unknown

Quotes on "Managers"

Management owns the 4 T's (time, talent , treasury, target )... everything else is up to the team. If you have a issue that affects one of the T's, then ask management for input. - Tim Ottinger

Message to Management - eventually, you can push too much change at once.... back off and let the team normalize. - JB Rainsberger

In my experience, it is the leader’s (manager’s) job to intervene in the case of inappropriate behavior, and when he or she does, the rest of the team will be grateful for the intervention. It is being a “servant” to the whole team by suppressing the individual behavior that will keep the team from being successful.

It’s not about telling people what they are doing wrong. It’s about constantly steering everyone on the team in the direction of success, and never letting any individual compromise the progress of the team toward success. - Mary Poppendieck

Note: where I can, I've credited or linked the source of the quote. Finding the source of a quote is like chasing a ghost. When a mentor says something witty, you might not know they are quoting someone else. If you are aware of a more appropriate source for any quote, PLEASE put a comment on the post and I'll do what I can to validate this. I mean no disrespect to anyone. I believe the risk of incorrect citings is outweighed by the value of sharing these wonderful nuggets.

Are Deadlines Important?

Olga Kouzina posted about deadlines on her blog here. She then posted a related discussion thread on LinkedIn's Agile and Agile Alliance groups. Next thing you know, there are 26 combined comments between the two threads!

She is basically questioning the core concept of deadlines. Do we need them? What value to they add? What negative impact do they carry? Unfortunately, some people mistakenly took this to mean that we should never work with goals, delivery dates, or time based measurements. I had to clarify my stance later in the thread with the following:

I believe in and agree with groups that leverage timeboxes, iterations, and release dates. I see these as good metrics and motivating goals.

I believe that Olga and I were talking about Deadlines with a capital "D". These are the dates put in the ground before the team has committed to scope, effort, etc. We know the requirements will change as we work the project, why won't the date change also?

I've seen too many projects where you realize pretty quickly that the "Deadline" was picked due to a desire but no real business need or planning validation. These dates are a great way to de-motivate a team... after all, why work hard if you already know you can't make it?

When Olga asked why I might think we put up with deadlines, I responded with:

We stick to them because people with higher authority don't trust a better solution and force them on us.

So... it's our job to start educating the decision makers around or above us. This is a topic I recently blogged about.

Ben Linders summarized the agile point of view well with the following:

If there really has to be a deadline, then it's up to the PO, project managers and stakeholders to do whatever they can to ensure that the teams know what to deliver, and have an enviroment and the right conditions in which they can be most effective and efficient. IMHO the deadline should never be passed on to the teams! The teams commits to iterative deliveries of highest priority user stories. That automatically leads to maximum result in the shortest time, which increases the change that deadlines are met.

And finally, I like Vinay Krishna's points:

Can we achieve anything without deadline? It looks funny to know that one starts a project and says that he doesn’t know when it would get finished.

I want to quote here the famous Parkinson Law which states that “Work expands to fill (and often exceed) the time allowed.”

This is important to recognize in software development because when we are not restricted by time and/or money, it’s easy to create an endless amount of features or strive for what we perceive as perfection.

Of course we can all agree there are sometimes market reasons for deadlines. After all some dates can't be moved (Y2K, elections, tax dates, etc).

Great stories...

If you haven't seen these great blog posts you should check them out:

Lean Team Software Process by Corey Ladas

snippet: An alternate metaphor for software value is a refined substance, like gasoline or ice cream. The raw material is customer requirements, domain knowledge, time, and energy; and this is transformed into manifest behavior of a computing system. This view suggests that software has no more customer value than the sum of the scenarios that it supports. Then there is no bright line that defines “complete”, there is only relatively more or less value delivered. When I fill up at the gas station, I need more than one gallon and less than 100 gallons, and when the pump tells me I have enough, I expect to pay the exact value of the utility I expect to receive from the product. Software can be delivered in this way: keep giving me more until I have enough, then stop and charge me for what I have consumed.

Metrics, Schmetrics II by Matthew @ Creative Chaos

snippet: All of those examples are fungible - a gallon of gas can be traded for any other gallon of gas. A penny saved is a penny earned. They are all the same.

Yet test cases, lines of code, these things are not the same. You can have a test case that takes two hours to set up, or a half-dozen similar ones you can run in thirty seconds.

Scrum vs. Kanban by Mike Cottmeyer (ignore the Idol intro and title at the beginning)

snippet: I think that in some ways Scrum is hitting a wall because some of us think that Scrum is the answer to everything. Is Kanban the new answer to everything? Is it going to replace Scrum and XP or DSDM or AUP or Crystal? I can't even imagine how we can have these conversations without understanding where and when is the best context to apply these principles. I feel like a broken record but there is a time and place for all these techniques.

Create a Proper Estimate Scale...

Tuesday I pointed at Jim Schiel's article about using an abstracted story point scale instead of calendar time (days/hours) for estimation. This morning I read a new article by one of his colleague's at Danube about their issues with their current internal estimation scale.

So, the question is, how DO you come up with a good abstract scale? Here are the main points I believe you need to address when attacking this discussion -

• Atomic unit of measure - the unit of measure needs to be atomic. This means that a value of 2 needs to be 2 times bigger than the value of one. A ten is 10 times bigger than the value of one. Why? so that when you know your team's velocity equals 10 and you select a 1,2,3 and 5 point story for a sprint, you know they actually add up to a sum velocity of 10.
  
• Restrict allowable numeric selections - don't allow just any old number to be assigned to a story. Poker planning estimation decks can't contain every number between zero and one thousand, and you don't want it to. Human brains are very bad at estimating. The larger the estimate, the larger the error. It is easy to state that something will take 1 minute to do... it is very hard to state with confidence whether something will take 60 minutes vs. 62 minutes. The higher your number, the wider the gap between confident values. Most teams use a doubling sequence (1,2,4,8...) or a Fibonacci sequence (1,2,3,5,8,13...). When estimating, you are not allowed to pick a number that doesn't exist in the sequence. Why? it is a waste of time to watch your team bicker over the 12 vs. 13 estimate value on a story when their estimation error is larger than the delta between the two sides of the debate.
  
• Create non-numeric selections - not all estimates can be covered by a value on your scale. Most poker planning decks include a few others. Examples: Question Mark (I don't understand enough to estimate with confidence), Infinity (I understand but it will take longer than we will ever have), Coffee cup (If I don't get a bio. break first, then my estimate will be whatever you want it to be), Zero (it can be done quicker than it will take to have this conversation). Why? This facilitates the conversations that planning and estimation need to flush out. Remember that estimation is not just about predicting, but also about understanding and committing.
• Pick a reinforcing name - pick a metaphor that will invoke interest and understanding. Many people call them points, but some use more interesting terms. Examples: headaches (bigger is bad), donuts (reward I'll get for delivering), dings (times I hit a bell when I deploy), dukets, dolla, goats, bricks, cans of jolt, etc. Why? It not only engages a team, but can put focus on a positive (or negative) part of the process that needs to be used as motivation.
  
• Allow adaptability - remember, this is an abstract scale. It can be adjusted. If after two weeks you find yourself using decimals because you need units under one... then shift your scale. I've done this twice and found two good options. Add a zero (multiply by 10) so that people don't lose reference to the old scale and can transition with ease, or make up a new scale and spend an hour quickly re-estimating the whole backlog. Why? When people have to think about what a value means, then they are losing their ability to do comparative estimation.
  

Feel free to add your comments about other concerns that need to be covered.

Note: If you are interested in other posts about why to use story points, I've also posted about it myself here and here.

2 great links...

• You get what you measure posted on Daniel Brolund's Weblog. It's a good factual background on a point I've made previously about not abusing your metrics.
• The product owner and the product-shaped hole posted on AgileProductDesign.com. Very thorough post about the role of an agile product owner or customer and how it is much more than stories and software functionality. Actually, it is a good follow-up to one of Jeff's past posts about the backlog.
  

Enjoy!

Don't abuse the metrics...

Tim Ross blogged about his concerns on burndowns. He offers some interesting points on how they can hurt including:

1. too much pressure early on
2. it doesn't handle the unplanned
3. it is unforgiving
4. PM's and customers misread it
5. it discourages refactoring
6. it focuses on a predetermined time

Hmm.... my responses:

1. really? most people feel it at the end
2. it is a reminder of the reality of the clock
3. neither is the market, it is changing whether you deliver or not
4. well... yes, that's true with any data point
   
5. not if you refactor constantly
6. or you could say it focuses on meeting a completion of commitment
   

Okay... some of that is me playing devil's advocate. I can relate to some of the points that Tim makes. Unfortunately, he proposes a solution where teams focus on the task board and maybe even hide the burndown for only the manager's eyes.

Umm... no. If you are really interested in going agile, then you have to understand that there is a power shift from the managers to the team. Instead of having people above (who aren't doing the work) ask for ridiculous things and make useless statements, the team is now more in charge of the process. But, the idea of a self-empowered, self-managed team is that the team has to be willing to be accountable for the work. This is not just the validation and quality of that work, but also the predictability of its timely delivery.

My view on this matter is two-fold:

• The burndown is an indicator. Bad trends in burndowns should be a flag for resetting expectations. When expectations are adjusted (be it the developers, customers, or managers), then everyone should know about it (that is why we do sprint planning and sprint review together, right?).
• ANY METRIC can be abused. If your culture stinks, or there isn't a trusting relationship between the team and the customer or managers... then any metric can be abused to support finger pointing. This is not specific to the burndown.
  

This is why many waterfall projects fail. It takes so long to uncover problems that when they are realized by lower management, they can delay the discovery further by gaming the metrics and reports to protect their jobs as long as possible.

"My definition [of agile] is that you accept input from reality, and you respond to it" - Kent Beck

A burndown is one of many measurements of reality! Unfortunately I think the answer in this situation is that the group needs to improve the cultural issues instead of throwing out the burndown.

Snake on the wall!

We found ourselves in a retrospective and there was a general feeling that there were a lot of interruptions in our day caused by "other people" outside the team. Nobody could put their finger on it and everyone had different unique examples. We couldn't easily say if we handle this problem or group of people then it will stop.

So how do you handle this?

Invoke the Snake on the Wall!

Every time a team member feels as though a task they are responsible for is delayed, they write it down on a post-it note. The note includes the time lost (compared to if they didn't have the delay), the thing affected, the cause, and their initials. They take the note and add it to the "end of the snake" which is a growing row of notes on the wall.

Over time the snake is monitored for repetitive patterns. The issues consuming the most productivity time are prioritized to the top of the list for the manager, scrum master, and team to focus on reducing or removing. Scrum masters look at it daily, managers look at it weekly (or when it grows suddenly), and the team reviews it right before retrospectives to trigger new topics. When an item is solved, those related stickies are removed from the snake and it is collapsed back down (you can't let it grow infinitely, otherwise you demoralize the team).

What does this accomplish?

• it validates real issues
  
• it kills false beliefs and misdirected complaints
  
• it quantifies the impact of impediments
• it creates transparency for managers who don't believe what you say
  
• it empowers the team
• it is immediate
• it self-prioritizes
• it uncovers surprises

On my last team we quantified for upper management how much time was lost EVERY DAY due to a bad source control system we were forced to use. It also provided insight into things we would never have thought were a problem. If used properly, it does a good job of quantifying the ongoing costs of certain types of technical debt.

So, I'm trying it a second time with my new group. We'll see how it goes.

Agile Risk Planning...

I'm going to oversimplify a lot of things in this post. I'm doing it for two reasons... it's not the most interesting topic, and I'm really just trying to throw out a very basic solution to what can be a very complex process. Many environments don't even do risk management, so my theory is that something light covering 60% of your needs is better than nothing at all.

What do we know about Risk Management?

Well, we are supposed to do our best to anticipate what might happen and mitigate it. The best way to do this is:

1. make a list of all the bad things that make you lose sleep (or just worry you)
   
2. assign a probability of each happening
3. assign a cost of impact if they occur
   
4. multiply all of the probabilities by all of the costs and get a total cost
5. put that money in the bank (or time padding on the project plan) to protect against it
6. constantly manage the list to your best knowledge
7. make decisions about how to spend resources to mitigate or remove risks (if you can do this for a fraction of the cost of it actually happening)

Your approach to this could be very scientific, thorough, and accurate... or you could just make some quick gut calls.

What is the agile solution I propose?

First, I believe that the amount of time needed to be invested in this process to make it very accurate is an amount of time and resources that most teams don't have. This is the same issue as estimation for stories and tasks in agile. There's a reason we do poker planning, and it think it might apply here.

1. Let's start with the basics... have the team spend a few minutes each release (every few sprints) brainstorming risks with the guidance of a PM and the PO. The PM, PO, and Scrum Master should have already pre-seeded the list to speed things up since this isn't the main focus for the team.
2. Now assume the risk will occur. Have the team estimate the impact of the risk using the same scale you estimate stories by.
3. Using poker planning, assign a probability of each happening (but use a scale of 1%, 10%, 25%, 50%, 75%).
   
4. Multiply each of the probabilities by its corresponding cost and sum for a total cost. This is your risk budget.
5. Here's where I assume you have an electronic version of your backlog (be it excel, or a tool like VersionOne). Add a story to the bottom of your backlog called Risks. Under it, create a task for each risk. Put your total risk budget as the story estimate.
6. Go.
   

Your system should include this story in any projections it makes for the burndown chart... so now you have that buffer built in to your projected product backlog. You have hedged your bets against the risk odds, and your are reasonably protected without being overly protective.

Responsibilities moving forward:

• Management is responsible for keeping this list up to date. Add/Remove risks as appropriate.
• Any work that can be done to reduce or remove items in this list at a fraction of the cost of the risk occurring should be pursued.
  
• The team should re-estimate both probability and impact cost periodically based on their updated knowledge (this supports the cone of uncertainty philosophy)

That's it... that's my poor man's soluton. Time to apply it and see if it is "just enough".

Final Note- Since I'm not as experienced in hand's on Risk Management as I could be, I welcome any ideas or feedback on this post... especially errors in my thinking. Feel free to beat me up. Most of the places I've worked, I haven't had time to focus on risk because the biggest issue is teaching people how to make and follow a plan along with communicating well as a team (hence my interest in agile).

Update (11/25): Here are two very good articles by James Shore about risk management and estimate inflation if you want to see a more in-depth look at these topics.

Story Pts vs. Capacity Hrs...

I was participating in the VersionOne user forums and a question was being asked about capacity. I'm guessing the person was a scrum master or project manager and they were asking about whether they should plan their resource ideal capacity for 8 hour days or 6 hour days.

This is a question that comes up from managers a lot, but it sometimes gets confused with velocity. I've seen people thinking that sprint planning is based on understanding a teams capacity and scheduling to it. Though there is some overlap and dependency between these two concepts... here is some clarification.

Philosophically, you want to use story points and velocity to plan your sprint commitment. You want to use hours and capacity to measure how you track towards that commitment throughout the sprint. Thus, you are better planning capacity at a realistic level. If for your group that is 6 hours, then plan 6 instead of 8. Teams that do support work in addition to project work typically have a lower capacity. This will vary by team.

In other words, hours shouldn't be used to plan and commit to a sprint. Remember that estimates are faulty and full of human error. The point of story points is to normalize this error out. But, using hours throughout the sprint to track tasks and watch trends is a good way to raise red flags and work with the team. This balance is part of the magic provided by the scrum master and project manager.

This is where the old style PM asks, "what about known vacation and holiday issues?"

Yes, you obviously don't let the team commit to the same amount of story points as the last 3 sprints (your velocity) if you know they are all taking 2 days off for Thanksgiving. But a simple rounding math formula can help you figure that out. Determine the percentage of capacity lost (rounded in days) and remove that percentage from your expected velocity for the upcoming sprint. That simple approach will probably be 85% accurate or more. Any more time spent figuring it out will not improve your numbers enough to be worth the investment.

Update (11/25): Mike Cohn just posted today along these same lines.

Is Burn-up more motivating?

Mark Hollman did a great job at describing the burn-up chart concept. I agree it is a great way to provide more insight to the team and whether their productivity vs. scope creep is having the greater impact on progress.

I did retort his point about motivation though.

My nugget: "... a burn-down goes to zero. It’s a floor that can’t be moved. A burn-up can keep going up. There’s no certain ceiling. It can be less motivating knowing that you are shooting for 50 today and tomorrow it’s 60. Zero is zero."