The Agent Web
Posts on how AI crawlers and agents navigate the web - what they see, what stops them, and what the next generation of web infrastructure needs to do differently.
Most writing about the agent web describes the future. These posts describe what is happening right now: the crawlers that run on yesterday's assumptions, the guidance documents that stop at the rendered page, and the non-English content that gets guessed at because the training corpus was never balanced.
The audience is developers, platform leads, and content owners who are deciding what to build next. The frame is practical: here is what the machines see today, here is the gap, and here is what changes when the gap closes.
Posts
What Google's web.dev Agent Guidance Does Not Touch
Google's 1 May 2026 web.dev guide tells developers to make their pages agent-friendly. The advice is sound. It also stops at the rendered HTML page. Provenance, authentication, rights, lifecycle, and off-web carriers are not in scope. MX is.
The Crawl Still Speaks English
Most AI models learn the web through one archive, and that archive is overwhelmingly English. Two years on, the share has barely moved. You cannot fix the model from outside, but you can stop it guessing about your non-English content.