Are your KPIs lying to you?

Confidence descends in the boardroom. The KPI dashboard is green: numbers are up, targets have been met. The management team is quietly satisfied. The business is performing well. The data says so.

 

Except, increasingly, the data is wrong. No one has falsified the figures, but the KPIs being reported are measuring the wrong things or presenting them in misleading ways. And they are giving leaders a picture of their business that their customers would barely recognise.

 

This problem is common across businesses, in sectors from financial services to retail, from logistics to the public sector. And it stems from three interconnected failures: choosing the wrong KPIs, reporting them badly and allowing the act of measurement to corrupt the activity being measured.

 

Measuring what is easy, not what matters

 

The most fundamental problem is KPI selection. When organisations design their KPIs, they tend to reach for what can be measured cleanly and quickly. That often means operational and process metrics rather than customer outcomes.

 

Consider a contact centre. It is easy to measure how quickly calls are answered. Five rings, ten seconds – the data is there, precise and unambiguous. So organisations set targets for call-answering speed, and report the results with pride.

 

What they may not measure with the same rigour is whether the customer’s problem was solved. The call was answered in two rings. The customer was then transferred four times, waited on hold for 11 minutes, and gave up. But according to the KPI, the interaction was a success.

 

This shows a failure to distinguish between activity and outcome. Activity KPIs, such as calls answered or deliveries dispatched, tell you that something happened. Outcome KPIs – complaints resolved on first contact; problems fixed within 24 hours – tell you whether it mattered. Most KPI frameworks are heavily weighted towards activity because outcomes are harder to measure and control.

 

Another common issue is the use of lagging indicators that track what has already happened, such as revenue, rather than leading indicators that predict future activity, such as proposals sent. Lagging indicators track what went well yesterday. But leading indicators tell you how likely you are still to be in business tomorrow.

 

Designing good KPIs is harder than it looks. It means avoiding vanity metrics that sound good but mean very little – metrics such as social media likes or website page views. It means looking beyond operational metrics into the customer’s experience. Genuinely customer-centred KPIs require detailed work: journey mapping, complaint analysis and a deep understanding of what can go wrong in customers’ lives.

 

Green dashboard syndrome: how averages hide the truth

 

Even when organisations have a reasonable mix of KPIs, the way those KPIs are reported can render them almost meaningless. A common culprit is the composite average score.

 

Imagine a business tracking ten KPIs: server uptime, processing speed, email response times, invoice accuracy and a handful of other operational measures – plus, at the end of the list, a customer satisfaction score. If the operational metrics are all performing well, scoring in the high 80s and 90s, and the customer satisfaction score is sitting at 40 per cent, the composite average across all ten might be 80 or 90 per cent. Green on the dashboard. Nothing to see here.

 

The problem is that operational and IT metrics tend to be precisely controllable in ways that customer outcomes are not. A business can almost guarantee 99.9 per cent server uptime with the right infrastructure. It cannot guarantee that every customer interaction goes well. When low-scoring customer measures are combined with high-scoring operational metrics, the measures of customer failure will be hidden.

 

The solution is not complicated: KPIs should be reported individually, not blended into composites. Customer-facing metrics should be given prominence in reports. And where averages are used, they should be accompanied by the distribution of scores because it’s the spread of scores that tells the real story.

 

The percentage trap

 

Alongside the averaging problem sits another issue: the comfort of small percentages.

 

A 5 per cent error rate sounds like a minor imperfection in an otherwise well-functioning operation. You can acknowledge it and move on.

 

However, a 5 per cent failure rate in a process used by 500,000 customers means that 25,000 people have been frustrated and dissatisfied. 25,000 customers you may have lost forever. Potentially 25,000 angry social media posts that could be taken up and amplified by the media.

 

This affects how leadership teams respond. Hearing that 3 per cent of customers experienced a billing error this month will generate a different reaction from hearing that 25,000 customers received an incorrect bill. The percentage shrinks the number to something that seems manageable. The raw figure tells a very different story.

 

The fix is simple. Every percentage-based KPI should be accompanied by the number it represents. Doing this means that leaders can understand the true scale of a problem.

 

Gaming the targets

 

There is a deeper problem with KPIs that leaders discover, painfully, on a regular basis: when a measure becomes an incentivised target, it stops being a good measure. For example, contact centres that measure average call handling time may find their agents ending calls prematurely, before customer issues are resolved, because a short call looks better on the dashboard.

 

The issue is not that targets are wrong in principle. It is that every target creates an incentive, and people are creative. The question any organisation should ask of every KPI is: if someone wanted to hit this number without actually improving performance, how would they do it? The answer to that question is almost always instructive.

 

Success with KPIs is in the execution

 

The irony is that even poor KPI design typically reflects a real desire to manage performance well. The impulse is right. The execution is where things go wrong: in the choice of metrics and the way they are presented.

 

Hitting a number and achieving an outcome are not the same thing. And until organisations understand that difference, their dashboards will keep telling them what they want to hear, rather than what they need to hear.