Auctions have a long history—even Herodotus, born 484 BC, mentions them in his Histories. But as fascinating as the history of auctions might be, a new story emerged in 1997 with the creation of GoTo, later Overture: a search engine whose model was to run auctions for rankings¹ . Today, most of the content you read online is subsidized by ads, and most of those ads are displayed based on an instantaneous real-time auction among competing bidders.

Allocating ads by auction, especially through some variant of a Vickrey auction in which the high bidder pays the second-highest bid, is basically a way to force ad buyers to show their cards. A bidder knows that they won't pay more than the next-highest price, and that bidding more increases their odds of winning. So the optimal move in the short term is to bid whatever price they'd actually be willing to pay. In the worst-case scenario, that is what they pay, but typically the amount is less.

This kind of auction creates a few interesting dynamics. For example, suppose there are several competing companies that all have similar cost structures. Let's say they can each turn a profit of roughly $2 per click in their widget-selling business, that they don't expect many repeat customers, and that they're using search ads as their main customer acquisition channel. Someone starts out bidding $0.05. A competitor tops them with $0.10. Then $0.15. Prices keep ratcheting up until everyone is bidding just under $2—and Google has eaten their entire margin.

Perhaps one of the widget companies changes their ad copy, plays around with pricing a bit, and discovers that they can earn $2.50. As soon as they start routinely ranking #1, though, the other widget companies will look at what they're doing. And, if it's a commodity business, they'll all promptly copy it. So, not only do the structural profits of the business flow to the ad company, but so do improvements in the business.

There's another case worth considering: what if one of the widget companies designs and patents a better widget, which its competitors can't match, and earns $3 for each click it buys. That company is the high bidder—but the next-best bid is $2.50, so that's what it pays. In other words, auctions reward differentiation, and punish commoditization² . Differentiated products lead to differentiated bids, and that’s what allows an advertiser to pay less than what a click is worth to them.

The other case is what happens when there's ad inventory but just one bidder. Or, equivalently, if there's ad inventory and the only bidders are those running poorly-targeted ads that try to make up for their lack of targeting with higher volume. Initially, running ads in this context is insanely profitable, and indeed the origin story of more than a few startups is "We tried AdWords/Facebook Ads when they first turned on self-serve, and we were paying pennies per click with basically infinite supply." As soon as other bidders move in, prices start to drift towards their natural level.

The term for this phenomenon, where ad prices reach the level the market will bear once there are multiple advertisers with good reasons to target a specific bit of inventory, is “bid density.” It’s an explanation for why monetization looks terrible in new products, and it’s why investors will sometimes make a leap of faith by comparing a new company’s ad revenue per active user (or ad revenue per user-minute, if they can get the data) to that of a more mature business. Bid density is the difference between giving nearly-free traffic to advertisers and capturing most of their margin.

Bid density has another second-order effect on how much an ad-driven platform can grow. Once there’s enough of it, there’s room to create more ads. Search ad products, whether within an e-commerce site or on a pure search engine, tend to grow for longer than investors expect because once there’s enough bidder interest to price two ad slots competitively, there’s often an interest in adding a third. And this process can theoretically continue until every organic result is an ad.

And this means that for the owners of ad-driven platforms, there are two strategic imperatives behind improving ad prices:

1. Whenever there's some industry with one advertiser who's doing extremely well on your platform, reach out to other advertisers in that industry and convince them to use your platform.

2. Do whatever is possible to commoditize companies so their click-level margins converge, ensuring that those margins accrue to the advertiser.

There are lots of reasons Google made Analytics free, but one of them was to ensure that everyone could get world-class analytics even at a low budget, and could easily track their search performance. The bet was that if a sophisticated company paid six figures for Omniture to make sure their ads were working, then there would be incremental revenue where that came from in ensuring that their scrappier competitors had access to a similar product. It's also a good reason for companies to improve their targeting tools; they don't want the best targeting to be available to the advertisers who achieve seven-, eight-, or nine-figure spend with agencies that can build tools internally; that money could go to the platform, instead!

And it's not just ads, either! Consider timeline recommendation algorithms. What they're trying to do is show you the content that's most likely to encourage you to keep reading, i.e. they're "bidding" a little of the time you've already decided to spend in the app, and hoping to convert it into more time later on³ . It's also one of the drivers behind the long-term economics for delivery services. When DoorDash signs up its first pizza place or sushi restaurant in a given city, it needs to rank that establishment first for the relevant food query. But once there are dozens, it can a) restrict the rankings to restaurants with high ratings, and b) still have enough of them that it can charge for placement⁴ .

In some ways, bid density is just a broader and more systematic implementation of the general observation that you'll be happier in negotiations if your next best option is still pretty good. Getting hired at a fair salary is a much simpler process when there are two companies making offers, while hiring decisions are also more straightforward when there are two people with equivalent skills. It pays to find ways to maximize your upside, and one of the ways that pays best, in ads and elsewhere, is to maximize the value of the next-best option.

Read More in The Diff

Bid density shows up frequently in The Diff, including:

• This note on how comping advertisers for their ad spend is a way to increase bid density ($).

• This writeup of Apple's growing ad business ($).

• This bit on food delivery companies ($).

• It even applies to raising venture money ($): if you try to raise at a time when there are fewer firms competing to lead the round, whether it's because of the general funding situation or because it’s August and decisionmakers are on vacation, you get significantly worse results.

1. The GoTo model managed to create both the search ads business model and the online ads arbitrage model; the company got traffic by buying cheap ads on other search engines and monetizing them better.

2. This is one reason some of the biggest Google search spenders have historically been companies like Booking and Expedia, which have laboriously compiled a roster of hotels for every conceivable travel search. Google would like it if there were, say, a dozen big online travel agencies, all operating at similar scale, and all with similar economics. But a duopoly can be profitable, especially because both companies are acquisitive, so when there's an emerging product that captures a new part of the market, one of them will buy it.

3. I sometimes wonder whether anyone will experiment with a market-based system for ranking content, where each page starts with a budget, bids for traffic, and then gets a larger internal budget based on the ad revenue it generates—including revenue from other pages that bid to be included as related content. Doing this would have an upfront cost and would add to the compute overhead for displaying pages. On the other hand, it's basically an implementation of the goal of every sufficiently well thought-out content recommendation system: maximize long-term value by continuously trading off between the content suggestions that monetize and the ones that keep users around.

4. There are other ways to get rankings, of course. Years ago, I was disappointed to see that my go-to Thai place, which had a name like "123 Kitchen Thai," had disappeared from GrubHub. It turns out that GrubHub had changed how they alphabetize, the restaurant had spotted it, and they were now "ABC Kitchen Thai" instead.

Help us learn more about you!

We'd like your feedback so we know what to cover, so if you haven't already, please fill out this one-minute form.