There's Never Just One North-Star Metric

Organizations need to measure their success, but there's a limit to how well they can compress this

Know someone who might like Capital Gains? Use the referral program to gain access to my database of book reviews (1), an invite to the Capital Gains Discord (2), stickers (10), and a mug (25). Scroll to the bottom of the email version of this edition or subscribe to get your referral link!

If you read enough corporate histories, you'll sometimes run into the electrifying moment where a company discovers that there's just one metric they need to maximize, and then gets busy maxing it out. That number is usually something related to usage—there's nothing special about realizing that revenue is really important, or that it's better for incremental margins to be positive than negative. But for many network effect-driven businesses, the insight that clicks is that every new user increases everyone else's usage by making the product more useful, and that this higher usage attracts even more users. This effect is trivially obvious in cases like messaging or payments, but it shows up in other things—file formats in the past, plugins and integrations for SaaS or even parallel computing platforms for semiconductors today.1

Once a company knows its metric, everything changes, because now every question can be framed as: Is this the best way to invest our money or our time into more signups/daily actives/user-minutes/gross merchandise value/whatever else? That tends to be immensely clarifying, and it makes these companies incredibly efficient because the internal arguments are very short—the big questions come down to uncertainty about how something will affect the One True KPI, or disagreements about the discount rate.2

That's a fun period: everyone knows what result they're gunning for, and they know how to measure it. They find the underlying models/inputs that drive it: "Seven friends in ten days" turned out to be the breakeven point for early Facebook retention, and getting that right meant having a smooth flow for inviting friends, and a comprehensive database of phone location pings that could tie two users together. (Spending a "getting-a-coffee-together" amount of time in the same place as someone else is a reliable way to get Facebook to suggest them as a friend, probably conditional on some existing level of social proximity like mutual friends or institutional affiliations.)

But it's also a short period. Any model is a compression, and if the thing it's predicting has more dimensions than the model itself, it's necessarily limited. Having a single KPI means projecting the whole space of optimal business practices into one dimension. That's a big deal if it works, because it takes a very differentiated company to pursue a very non-specific goal.

As soon as the company's having a real-world impact commensurate with their achievement of that metric, they suddenly find that there was a long list of offsetting metrics that they needed to incorporate into their model: any kind of scale economics leads to the suspicion of monopolistic behavior, and if there are network effects involved then monopolistic behavior is the involuntary default. Meanwhile, taking a messy reality and grading it according to a single metric slowly selects for the easiest-to-fuzz version of that metric: not all daily active users are of equal utility, and some of them are active liabilities.

So the era of one key metric might best be understood as a final adolescent growth spurt. There's an intoxicating sense of having finally zeroed in on the core problem to solve—and then, as it's solved a growing fear while it was a good metric, it wasn't the only thing that mattered.

Understanding businesses means understanding what they're measured by and why:

Share Capital Gains

Subscribed readers can participate in our referral program! If you're not already subscribed, click the button below and we'll email you your link; if you are already subscribed, you can find your referral link in the email version of this edition.

Join the discussion!

1  Oddly enough, social networks are a case where the first insight usually needs to be about a restrictive model, rather than an open one: these networks thrive when there's a group of people who default to a particular app for a certain kind of update, whether that's telling LinkedIn about a new job or informing Snap that they're single. They only want to go for maximum scale once they've saturated enough high-value real-world networks to be a clearly high-status option.

2  One of my favorite examples of this is in Chaos Monkeys, where the author, who worked on ads at Facebook, has the insight that the Facebook logout page is one of the highest-traffic webpages on earth, and could be a great place to put an ad. As it turned out, this page was also one of the most valuable ads in the world already, since most developing-world Facebook users logged in through Internet cafés and would leave the logout page up when they left. Fortunately, a compromise suggested itself: run the ads in mature markets, where the CPMs were higher, anyway, but not in developing markets. Cheap smartphones made the whole thing a moot point a few years later.

Reply

or to participate.