• Capital Gains
  • Posts
  • The Share of Bots to Humans is Rarely a Meaningful Metric

The Share of Bots to Humans is Rarely a Meaningful Metric

LLMs may dominate pageviews, but they're still working for humans

Sponsored by

Know someone who might like Capital Gains? Use the referral program to gain access to my database of book reviews (1), an invite to the Capital Gains Discord (2), stickers (10), and a mug (25). Scroll to the bottom of the email version of this edition or subscribe to get your referral link!

Last month, Cloudflare's CEO announced that bot traffic had overtaken human traffic. For the first time ever, most of the time a page gets accessed online, it isn't being seen by human eyes. Which feels disconcerting—are we already in some posthuman-by-a-small-margin future? By that metric, yes, but that metric also gets less meaningful as the number of bots rises.

An easy way to understand this is to look at spaces with a higher bot-to-body ratio, like equity trading or display ads. It's hard to get exact numbers, but if you think of all of the trades that are purely systematic, or index funds (basically a systematic strategy, though the algorithm is just "own the following 500 or so stocks in the following proportion"), or big trades systematically chopped into tiny, lower-market-impact lots, whose counterparties will be heavily automated market-makers. A single decision to buy 10,000 shares of some stock might produce dozens of 100- and 200-share lots, with most of that selling done by systems that are automated at the level of deciding what spread and size to quote. So, in one sense, that set of transactions is 90%+ automated. But it's intermediating between someone who made the discretionary decision to add 10,000 shares, the human software engineers who built the trading system, the traders who supervise it and tweak its parameters, etc. All that autonomous behavior is just intermediating between human agency, once the decisions those agents make are clear enough that bots can handle the details.

Online advertising, outside of walled gardens, has a similar dynamic.1 If you see an ad on some news site, the publisher, contextual data provider, yield manager, multiple sell-side platforms, an ad server, a demand-side server, and a CDN for actually displaying the creative will all be involved. Plausibly, up to a dozen systems will automatically make a decision based on the decision to view a single page (though in some cases, that decision is basically IF user_views_page THEN show_ad). But, as before, there are many humans in the loop, just at different layers of abstraction. If the ads aren't showing up, the publisher has someone they can yell it; if they are showing up in inappropriate contexts (e.g, an ad for cheap flights running on a news story about a plane crash), the advertiser also has someone to talk to (though various elements of this moderation process are automated in some contexts as well).2

The share of bot actions to human actions is really a measure of at least three variables: the fraction of people who have adopted some kind of automation tool, the fraction of tasks that they can hand off to these tools, and how thorough the tools are at their jobs. A fairly simple decision, like "let's bump up our display budget by another 10%" or "I'm feeling nervous about Iran, better buy some index puts" will kick off a frenzy of bot activity, only a tiny fraction of which is directed manually by humans.

In the case of LLMs, a lot of what's happening is that reasoning models fan out very broadly (and sometimes seem to get caught in weird diversions—or take a couple tries before they find an un-paywalled version of the obscure PDF with the answer to your query). When they do this, part of what they're doing is substituting a relatively simple lookup for a more complicated inference task; if you ask a model how quickly a Model Y goes from zero to sixty, the model could look up detailed specifications of the car and then model the physics of acceleration, or it could find some brief description where someone else has done the relevant work. All this bot traffic is, relative to some adjacent possibilities, actually reducing datacenter demand.

It's strange that something that started as a default-human behavior—accessing some URL and interpreting the content there—is now automated. But in a sense, that's what's been happening to the economy for a very long time. The supply chain that keeps a good car in working order is vastly more complicated than the supply chain that keeps a horse functioning, as evidenced by various episodes in which steppe nomads conquered important places and held on to them for a while. But the effort required from an owner of a car is far lower. The same is true for computers (we're a long way from the "bad command or file name" days, and can now just blindly paste error messages into our chatbot of choice and navigate to a fix). Many complex, multi-step tasks get encapsulated in some simple kit that does the hard parts remotely and doesn't ask the user to do anything that can't be explained through pictographs.

Broadly speaking, this is a necessary and useful feature of modernity: specialized cognition gets done by specialists, and the average consumer gets a very simple experience by default and is opting in to anything that requires more than a middle school education. It actually creates a strange dynamic where the simpler an interaction with some product is, the more dizzying the supply chain behind it—what could be more natural than asking a question in natural language, and what consumer experience taps into as much cumulative capex as using an LLM chatbot for that?3 A rising ratio of bots to humans implies a field that's getting closer to being solved, but demand is unbounded, and a field that gets cheaply solved is one that absorbs a lot of incremental consumption. So, expect any ratio of this kind to rise over time, and expect it to be a disconcerting feeling when it does.

The Diff has looked at the layers of automation between human decisions and machine activity from a few angles:

Expense receipts shouldn't require a search party

Adam spent 20 minutes looking for a $36 receipt. His finance team sent three Slack messages. Someone made a sticky note.

Ramp would have matched it automatically the moment he swiped. Auto-coded, in-policy, synced. Nobody had to ask Adam for anything.

This is what finance looks like when it runs itself.

Your team can be Adam. Or they can not be Adam.

Share Capital Gains

Subscribed readers can participate in our referral program! If you're not already subscribed, click the button below and we'll email you your link; if you are already subscribed, you can find your referral link in the email version of this edition.

Join the discussion!

1  Inside walled gardens, the supply chain is simpler and everything ultimately rolls up to the same P&L, but a given displayed ad is still going to touch many different systems, potentially indirectly, but in a more opaque way.

2  There are some industries that are basically in the business of taking a fragmented process and giving customers—some of whom like to use this precise terminology—one throat to choke. An ad agency can commit to some set of advertising standards that compile down to display ad plumbing details that their client in the detergent or mortgage or whatever industry aren't familiar with.

3  Probably anything involving hydrocarbons is up there, and when you pay for your chatbot, you're also interacting with a multi-trillion dollar campaign to make it possible for complete strangers to exchange money for goods and services without a second thought about whether or not their counterparty will default.

Reply

or to participate.