The Metrics Question

measureLarge organizations live and die by the metrics they establish. Metrics drive bonuses, rewards, and punishments and impact all organizational behaviors.

Every time we work with a new group of executives we always hear the metrics question. Specifically, “What new metrics do we need to support this change?”

It is a loaded question. Metrics have extensive power to drive behavior.

Hiding the Bodies

Brazil was selected to host the World’s Cup in 2014. The problem was Rio de Janeiro had a very high reported murder rate. This is bad for business. Something about a high murder rate discourages tourists.

The government established a metric goal for the Brazilian security forces—get the murder rate down!

The security forces rose to the challenge and over the course of a year Rio de Janeiro’s murder rate dropped dramatically. The initiative was a success. The metric was a success. The method was interesting.

HBO’s documentary ‘The Pacification of Rio’ reported that the murder rate dropped dramatically before the games because if the Brazilian security forces encountered a body they simply made it disappear.

No body, no murder to report. The murder rate drops.

The missing person rate increased. But apparently World Cup tourists don’t pay much attention to that statistic.

“What Gets Measured, Gets Done”

The metrics trap is that metrics work. They really, really work. What you measure and reward actually gets turned into human behavior. If you establish a metric, even a great sounding metric like lets decrease the murder rate, your staff will do whatever it takes to get the desire result. Even if their behaviors are counterproductive to the real goal.

The security forces were being handsomely rewarded on their ability to decrease the Murder Rate.
So they did whatever they could to decrease the number of murders reported.

Metrics are nefarious. It is hard to determine in advance exactly what creative, innovative, and intelligent people will do to meet the metric. This you can know for certain… the more metrics you put in place the more your people are hiding the bodies.

Bad Metrics

Most metrics are bad. They are bad because they work against the true goal. They are bad because they result in unexpected and even bizarre behavior. Noble intentions do not guarantee noble outcomes.

Having your security forces steal bodies off of the street before the local homicide investigators can get there is certainly an unexpected behavior. Having them bury the body in the forest or dump it in the ocean is definitely bizarre.

Best of all, hiding the bodies almost guarantees the killer will kill again. Which works against what should be the real goal.

This is the problem with almost all commonly used metrics. If we closely observe the effects they  produce we rapidly discover ways they work against the real goals. Of course, the numbers look great!

Trust only Movement

The Agile answer is to make the entire operation as transparent as possible. Then observe the actual behaviors. The behaviors matter. Metrics can always be gamed. The only way to really understand what is going on in your organization is to observe it in action—trust only movement.

Agile is deeply aware that metrics create counterproductive behaviors. Agile transformations begin by eliminating most or all of the existing metrics. Sure there are a few things worth measuring. We list four below:

  1. User Satisfaction
  2. Business Value
  3. Cost of Change
  4. Employee Satisfaction

Focus on a few simple and visible metrics. Watch the process. Ensure you are actually getting the results you really want, not just the results the metrics measure. Abandoning blindly steering and managing by metrics.

Only by abandoning most of your metrics will you ever truly be agile.

By Tom (Thomas) Meloche –

If you have 1,000 software developers you need at least 10 to 12 agile tribes. Agile tribes are limited to about 100 members because this seems to be the natural limit on building a community capable of maintaining agile standards socially without requiring a bureaucracy. One hundred people is large enough…
If the cost of business change is high we've lost our ability to embrace change. We've lost our ability to be innovative and responsive. This is why understanding your cost of change delta is critical. You need to get your cost change curve as flat as possible. The cost of…