The web is splitting in two. Software developer Vincent Schmalbach discovered that TIME magazine now serves two versions of the same website: a full HTML experience for humans, and a stripped-down markdown copy — with ads baked in — for AI bots.
The Experiment
Schmalbach fetched an ordinary TIME health article repeatedly from the same machine, changing only the User-Agent header. As Chrome or Safari, he received 303,235 bytes of HTML. As Googlebot, the same. But as ClaudeBot, PerplexityBot, or OpenAI's OAI-SearchBot, he received 13,409 bytes of text/markdown — one twenty-third the size — with no layout, no images, and no human reader, Schmalbach reported on August 5.
The Hidden Ads
The markdown pages carry a comment and an x-mobian-tokens header that counts tokens fed into a model. Sponsored content appears on list and section pages: one markdown fetch of TIME's "Best Inventions of 2025" returned a full Ally Bank FAQ and a Project Management Institute "Reference Facts and FAQ" table that do not exist in the human HTML. The ads are labeled as sponsored, but the audience split is invisible to human readers.
Selective Bot Policy
Not every bot gets the markdown treatment. GPTBot and ChatGPT-User are blocked with a 406 response, while OAI-SearchBot — the crawler that feeds ChatGPT's search index — is waved through. Googlebot sees the same HTML as a human, so search rankings are unaffected. TIME is making per-bot decisions about who sees what.
Why It Matters
Schmalbach notes that TIME has said bot traffic already outnumbers human traffic on most days. The experiment is one of the clearest looks yet at a future where publishers optimize a layer of their site entirely for machines — and where the line between content and training fuel becomes hard to see.
Comments (0)
Log in or sign up to leave a comment.
No comments yet. Be the first to share your thoughts.