The exciting promise of the early internet was that instead of dealing with a maze of intermediaries, we could skip all of those extra costs and just buy directly. No more travel agents, insurance agents, or real estate agents; retail would be transformed into a way to buy goods directly from the people who made them without giving Target or Walmart a piece of the action; we'd be able to trade directly with one another rather than going through third parties.

And even though that sounded reasonable—even though making information cheaper, more abundant, and easier to sort through does mean it's more straightforward to go directly to what you want without paying some intermediary a markup—the result of that economic revolution is that three of the top ten companies by market cap are middlemen, and two of the remainder, Nvidia and TSMC, get a lot of their business and an even more disproportionate share of their growth from the spending of those intermediaries.

And these intermediaries, who took share from older retailers and media companies, tend to report higher margins and have far more market power. That's true in smaller industries, too; airlines and hotels have consolidated, but Expedia and Booking Holdings are collectively worth over $180bn. Even reducing the number of end providers in a product category didn't eliminate the economic value of aggregating them.

Economic intermediaries exist because of search costs: customers have some parameters for what they're looking for, and companies compete for their business, but someone needs to match these needs to the businesses able to provide for them. The utility of that matching is a function of, roughly, how hard it is to find the specific thing customers are looking for, how little various options are differentiated, and how much the end customer values getting exactly the right thing versus something close to it.¹ But search costs can be paid by consumers, by manufacturers, or by some intermediary. Consumers mostly pay search costs in time, though they'll sometimes subscribe to publications that are mostly focused on telling them what to buy, ranging from Consumer Reports to Vogue. And companies run ads, or just aim to have an exceptionally trustworthy brand. When that search process gets cheaper, it should mean that more entities do it, and that is indeed the first-order effect.

But the second-order effect is that if the cost of search is lower, the benefit to creating a searched-for product is higher. As markets get defined more narrowly, there are more micro-markets to dominate; even if there's consolidation happening at the company level, there's still proliferation at the brand level.

So the task of intermediaries gets more challenging; there aren't just more products to evaluate, but more axes to evaluate them on, and that means more categories of customers to track or more data collection needed to train models that cluster them automatically. And that has two related effects: first, it means that winning the race to make the right recommendation is more rewarding, and as a side effect of that, there are returns to scale. A bigger social network can analyze more connections between users to see how purchase trends spread; a bigger search engine has more clickthrough data on what people actually wanted to see when presented with multiple results. They win enough to scale and then scale to the point that winning is inevitable.

A world where information is cheap is a world where acquiring new preferences and opinions is extraordinarily cheap, and where launching new products to cater to this is not an onerous task, either. In that world, the absolute advantage of companies going direct is higher, but the relative advantage of selling through middlemen is higher still. All of those options create a cacophony of consumer possibilities, and most consumers simply don't want a multi-worksheet Excel model to be part of their process for choosing their next tube of toothpaste. A sufficiently large e-commerce platform, social media site, or search engine is implicitly running an even more elaborate model to make this same decision for their customers.

As it turns out, this was the big effect of the combined hit of the Internet and globalization—one trend that made more information accessible faster, and another that spawned more adaptive global supply chains that could rapidly respond to changes in consumer demand. It is still trivial to go direct, for both consumers and businesses (in some circles, it's practically a sport to see how fast you can find an Amazon product listed at a fraction of the price on Alibaba)—but what you get when you go through an intermediary is so much closer a match to what you wanted that it's worth the extra cost.

Read More in The Diff

The Diff has written extensively about intermediary businesses, including:

• We can’t really afford the ownership economy.

• Platform companies function like local politics ($).

• Some companies, like HBO, make the transition from pure distribution to owning what they sell ($).

• One reason platforms grow is who grows alongside them ($).

Join the discussion!

1  All of these are well-illustrated by the recruiting business, which tends to charge substantial fees—typically 20% of annual compensation, but sometimes more than that—for a successful placement. If you view it as a payment for hours worked, recruiters are fantastically overpaid, at least for the successful placements. But if you compare it to the opportunity cost of sorting through candidates, it can be cheap: even phone screens have an opportunity cost, and the better-filtered candidates are, the more of a company's recruiting time gets invested in vetting people they're probably going to hire instead of the ones they won't. Meanwhile, the recruiter is evaluating people for their fit across a broader range of firms—the same candidate who's a "no" because they want remote work, or prefer a different tech stack, or need more salary in the compensation mix, might be a fit somewhere else. So it's optimal for recruiters to cast a broader net than individual companies.