“What Gets Measured, Gets Done”

Brazil was selected to host the World’s Cup in 2014. The problem was Rio de Janeiro had a very high reported murder rate. This is bad for business. Something about a high murder rate discourages tourists.

The government established a metric goal for the Brazilian security forces—get the murder rate down!

The security forces rose to the challenge and over the course of a year Rio de Janeiro’s murder rate dropped dramatically. The initiative was a success. The metric was a success. The method was interesting.

HBO’s documentary ‘The Pacification of Rio’ reported that the murder rate dropped dramatically before the games because if the Brazilian security forces encountered a body they simply made it disappear.

No body, no murder to report. The murder rate drops.

The missing person rate increased. But apparently World Cup tourists don’t pay much attention to that statistic.

The Metrics Trap

The metrics trap is that metrics work. They really, really work. What you measure and reward actually gets turned into human behavior. If you establish a metric, even a great sounding metric like lets decrease the reported murder rate, your staff will do whatever it takes to get the desire result. Even if their behaviors are counterproductive to the real goal.

The security forces were being handsomely rewarded on their ability to decrease the Reported Murder Rate.
So they did whatever they could to decrease the number of murders reported.

Metrics are nefarious.

It is hard to determine in advance exactly what creative, innovative, and intelligent people will do to meet the metric. This you can know for certain, the more metrics you put in place the more your people are hiding the bodies.

Bad Metrics

Most metrics are bad. They are bad because they work against the true goal. They are bad because they result in unexpected and even bizarre behavior. Noble intentions do not guarantee noble outcomes.

Having your security forces steal bodies off of the street before the local homicide investigators can get there is certainly an unexpected behavior. Having them bury the body in the forest or dump it in the ocean is definitely bizarre.

Best of all, hiding the bodies almost guarantees the killer will kill again. Which works against what should be the real goal of ACTUALLY DECREASING MURDERS.

This is the problem with almost all commonly used metrics. If we closely observe the effects they produce we rapidly discover ways they work against the real goals. Of course, the numbers look great!

Trust only Movement

A better answer to a lot of metrics is to make an entire operation as transparent as possible, then observe the actual behaviors. The behaviors matter. Metrics can always be gamed. The only way to really understand what is going on in your organization is to observe it in action—trust only movement.

Methodologists are deeply aware that metrics create counterproductive behaviors. Hence, in most transformations I lead, we begin by eliminating the existing metrics and starting afresh with the vital four. I will review these one at a time in four different posts. These, coupled with transparency, are a great place to start.

 

 

“I need a name for something,” I told Kyle, as we sat down to a Barbecue dinner. “What is the problem?” Kyle asked. “You know how at Menlo we radiate stories, status, and risks on a task board next to the team?” I said between bites of potato salad. “Go…
I attended a presentation by one of the top agile metric gurus for a leading Agile tool vendor. Their tool tracks user stories, tasks, points, hours and generates charts and reports. Because over 35,000 “projects” have been captured with the tool the company did a great deal of analysis across…

Be the first to comment

Leave a Reply