Are North Star Metrics Inherently Flawed

The idea of a North Star Metric is seductive.

Originating from the tech crucible of Silicon Valley, they are supposed to represent a radical approach to growth strategy, promising a navigational beacon through the dense fog of data. 

The concept is straightforward: identify a single metric that encapsulates the core value you deliver to your customers, and then align all your efforts towards improving it. 

When I first came across the idea a decade ago, it was like a breath of fresh air. My early career was largely spent building shrines to vanity metrics; I craved greater meaning. The ringing endorsement of companies like Facebook, LinkedIn, and Airbnb only added to the allure.

‍
As with so many things however, theory is one thing, practice is another. I tried implementing North Star Metrics twice (from memory); once at a budding startup, and once at a larger enterprise.

Despite initial enthusiasm, the simplicity often boiled down to reductive analysis that didn’t translate into meaningful improvements. The initiatives, though well-intentioned, eventually faded into the background.

Goodhart’s Law

Implementation aside, I’ve come to believe North Star metrics are a flawed concept from the outset. Consider that on both occasions:

• The anointed metric was demonstratively correlated with the value we provided to customers
• We were able to map out a bunch of factors that affected it, relating to various initiatives and/or parts of the business

The problem was that the correlation seemed to weaken, or break down entirely, whenever we tried to improve those contributing factors. Why? I think the short answer is Goodhart’s Law.
‍

Essentially, when you optimise for a specific metric, its ability to accurately reflect value diminishes. The classic example involves nails:

It's not hard to find examples of this dynamic playing out all around us. Teachers often lament that defining the success of a school based solely on test scores will inevitably lead to obsessive test preparation, potentially at the expense of broader educational value.North Stars are similarly prone to perverse incentives. Let’s say you set “number of resolved customer issues” as a key contributing factor to customer satisfaction, and start attaching KPIs to it.

Assuming CX agents are already doing a diligent job, they may start prioritising closing tickets over actually resolving problems. If you start specifying “positive resolutions” in response, the bar for what counts as positive will inevitably start to creep down… and so on.This doesn’t mean that targets are bad per say, only that the drive to hit numerical targets can lead to a degradation in the quality of service, as the metrics become more about meeting quotas than delivering real value.PPC algorithms are extremely prone to these traps.

I was recently contacted by a colleague of a friend who was struggling to understand why their hugely popular newsletter was flatlining, despite a raft of acquisition spend on Facebook. Their cost per subscriber was low, but their open rates were falling. Cue much wailing and gnashing of teeth... The slightly depressing answer was that Facebook had found them a bunch of users who love signing up to newsletters, rather than people who were genuinely interested in the subject matter.Similarly, a client of ours once figured out that users who completed their (necessarily) long sign up flow were very likely to make a purchase, so told Google to optimise for the sign up event… I’m sure you can see where this is going. Suffice to say purchases did not increase.

‍
In conclusion, the appealing simplicity of North Star metrics is somewhat undermined by their vulnerability to “Goodharting”. The idea that a single metric can guide a company to success is often overly reductive, potentially misleading, and sometimes just painfully obvious.

‍AirBnB’s most important metric is “number of nights booked” - is that really a revelation? Worse still, did Facebook’s focus on “time spent on platform” make the service more valuable for its users, or its shareholders?

If you’re struggling to move beyond vanity metrics, North Stars can be a useful gimmick. For everyone else, I would suggest broadening your horizons, and navigating by a constellation of metrics that together offer a fuller picture of success.