Every Machine Says Its Own Name
Seven crawler names reached this site in seven seconds, and what they asked for was the file where the keys live. The permission layer of the web runs on a text field the visitor fills in himself.
On 21 August, between 15:50:49 and 15:50:56, seven crawler names reached this site: CCBot, Vertex-Bot, GPTBot, PerplexityBot, ChatGPT-User, xAI-SearchBot, DeepSeekBot. Six companies, seven seconds. That morning five other names had opened their first request within thirty-eight milliseconds of one another. Across the day 513 requests arrived from Singapore under names that have never once come from Singapore on any other day.
What they asked for settles what they were. ClaudeBot requested /@fs/root/.aws/credentials. OAI-SearchBot requested /@fs/home/ec2-user/.aws/credentials, ec2-user being the default account on an Amazon Linux machine. Google-Extended requested /.env. Amazonbot worked through fourteen variants of an environment file, then /.bash_history, then /.git-credentials, then /.claude/settings.json, which is where an AI agent keeps its keys. That /@fs prefix is the signature of CVE-2025-30208, a file-read hole in the Vite development server. No crawler reads /.env.production.bak. One wordlist was spread across six companies' names.
A name in a header is not an identity. The User-Agent is a free-text field, written by whoever makes the request and checked by nobody.
The protocol says as much in its own text. RFC 9309, which standardised robots.txt in September 2022, states that its rules "are not a form of access authorization", and its security section adds that the protocol "is not a substitute for valid content security measures". Three of its four authors work at Google. The people who wrote the lock wrote down that it is a sign.
Cloudflare measured the same gap on 4 August 2025, on test domains whose robots.txt disallowed everything. It reported that Perplexity's declared crawler stopped while traffic continued from an undeclared one, a generic string impersonating Chrome on macOS, rotating the networks it arrived from. In the same tests ChatGPT's crawler fetched the file and obeyed it. Both behaviours were available to both companies. The file could not tell them apart.
Then Cloudflare built the replacement, which is the part worth reading twice. Web Bot Auth arrived in May 2025 and Signed Agents on 28 August 2025: a crawler signs its requests with a cryptographic key, and the signature travels in the headers where the name used to be enough. The company with the widest view of crawler traffic spent a year building an identity layer. That is what you build when the name is worth nothing.
My own log carries a column for whether Cloudflare recognised a visitor as a verified bot. It holds 3,191 rows and every one of them says no, because the plan this site runs on never populates the field. The register I own records what each visitor called itself, and stops there. An agent is, before it is intelligent, a change in permissions, and the permission here is granted to a string.
Take those days out and my own numbers change shape. ChatGPT-User, the count I had been reading as evidence that a person handed one of these pages to a machine, drops from 227 to 18. PerplexityBot drops from 69 to 6. Google-Extended, the crawler that decides whether Google may train on this site, drops from 55 to zero, which means it has never been here at all. Around sixty per cent of what my dashboard called AI traffic was one scanner wearing their names.
I keep a page for machines on this domain. It says who publishes here and asks a crawler to attribute properly. It believes every visitor about what it is, because there is nothing else to believe, and after this month that makes it the most honest object on the site.
Certainly voluntary declaration built the crawlable web, and it still mostly works: the well-behaved crawlers announce themselves, fetch the file, obey it. Anti-hype is not anti-everything. What has changed is the prize. A crawl that fed a search index sent traffic back to the site it took from. A crawl that feeds a model sends nothing back, and the value of not being caught rises with it.
Every regime built this year rests on the same field. The copyright opt-out, the AI Act's training-data summaries, robots.txt, allow-lists, the crawl controls now sold as governance: each asks a machine to say what it is, then acts on the answer. Content marking has the shape from the other end, where the mark names a tool and says nothing about a person, which is why it reads as a receipt. Who signs? On the open web, still nobody.
Every machine says its own name. The name is free to write and free to change, and everything downstream of it was built as though it were not.