Not Everything That Counts Can be Counted

Every business needs to gauge its performance. Metrics such as revenue, profit, employee count, and market capitalization are how we compare and assess every business. In recent years, there has been a surge in business metrics across various functions, measuring everything from marketing pipeline and customer satisfaction to R&D defects. 

The obsession with metrics reached its crescendo following the release of John Doerr's book, "Measure What Matters," in 2017. Mr. Doerr asserted that business leaders should measure everything that matters and disregard everything else because if it can’t be measured, it can’t possibly be important. Because he’s done very well for himself, he must be right, no doubt. 

After delving into Mr. Doerr’s book, numerous companies and consultants eagerly embraced the OKR (Objectives and Key Results) methodology, supplanting KPIs and other previous metric fashion trends. CEOs, inspired by the likes of Google, Netflix, and Amazon, mandated the adoption of OKRs across all departments, converting most meetings into KPI review sessions in which their teams try to interpret the data. 

While utilizing data for decision-making is a must, it poses some challenges. Firstly, data quality is often compromised due to measurement difficulties and human tendencies to fudge it...to look good in those review meetings. Secondly, misinterpretation is common, leading to confusion about causality and overlooking the bigger picture. For instance, in A/B testing, the focus on which alternative performs better might miss the possibility that both alternatives are bad (or that they are both great and we shouldn’t dismiss one of them).  

The third issue with running a business by metrics alone is that not everything is measurable. It is dangerously naïve to dismiss this argument by saying that whatever can’t be measured cannot possibly be important. That’s just wrong. We haven’t figured out yet how to measure creativity or subjective quality. We can’t quantify strategy. We can only guess the likelihood that a plan will work. We don’t know how to measure leadership. There is no metric for user experience. These are all examples of critical aspects of business that are not measurable. 

Sure, we can try to measure these factors using feedback, approval rates, and clicks, but they often yield imprecise and biased insights, capturing a limited perspective. The basic question “Is our product any good?” is not answered by the count of priority A defects. The question “How strong is our brand?” is not answered by the number of LinkedIn followers. And the question “Why did we miss our revenue target?” is not answered with pipeline conversions data. These data points may serve as interesting proxies but just because they can be measured, they don’t give us the full picture. 

Any decision in such unmeasurable areas has to rely on experience, judgment, and talent. Unfortunately, these decisions are frequently entrusted not to the individuals with the right expertise but to their managers, usually one or two levels above, who potentially lack the full context required to make the call. BTW, one of the greatest opportunities for AI is right here - let the AI figure out how to make these decisions.

Further, because these unquantifiable areas don’t fit into a dashboard format, very few discussions tend to be devoted to them. Some companies can spend way more resources on analyzing their win rate than on analyzing the strategy that drives the win rate. They spend more time on measuring Sales enablement than on actually enabling the sales reps. They spend more effort on analyzing NPS results than on talking to customers. Focusing on what is measurable can lead to avoiding the real problem. 

I am by no means suggesting that metrics don’t matter. When we have good metrics, we would be foolish not to use them for making well-informed, data-based decisions. But making decisions based on good data is easy. The tough decisions are the ones that have to be made with little or incomplete data. We cannot neglect the non-measurable areas of business just because we can’t track them on a dashboard. 

After all, the great Albert Einstein once said: “Not everything that counts can be counted, and not everything that can be counted counts.”