Somewhere in your server logs, between the Googlebot hits and the people you built the site for, there is a new kind of reader. It identifies itself as ClaudeBot, or GPTBot, or PerplexityBot, or one of a dozen quieter names. It doesn't scroll. It doesn't wait for your JavaScript bundle. It reads whatever your server actually sends, decides in milliseconds whether your site answered its question, and moves on.
That reader is there because a human asked it something. "Find me a good accountant in Graz." "Which of these two tools is better maintained?" "Summarize what this company does." The answer the assistant gives - and whether your site is the source it quotes or the one it never saw - depends on things most websites have never been tested for.
That is what we mean by AI readiness, and it is why we grade it on every audit now, right next to performance and security.
The part that stings
Here is the uncomfortable bit: a site can be beautiful, fast, accessible, and rank fine on Google - and still be nearly invisible to an AI assistant.
The reason is mundane. Most assistants read the raw HTML response. If your content only exists after a framework boots up in the browser, the assistant gets a loading spinner and a <div id="app"></div>. It won't guess what would have appeared there. To that reader, your carefully written pricing page is an empty room.
We see this constantly in audits. The marketing site renders server-side and reads fine; the docs, the changelog, the product pages - the parts people actually ask assistants about - are client-rendered and come back blank. Nobody chose that outcome. It's just what happens when nobody looks.
What it actually takes
The good news: none of this is exotic. AI readiness is six mostly boring things done properly. These are exactly the six checks in our audit, so when your report shows a grade, this is what stands behind it.
1. Content that exists before JavaScript runs. The single biggest factor. Server-render your pages, or pre-render them at build time - whatever your stack calls it. The test is brutally simple: fetch a page with curl and look at what comes back. If the words you care about aren't in that response, most machine readers never see them.
2. An llms.txt file. A proposed standard, young but spreading fast: a markdown file at /llms.txt that tells AI systems what your site is and where the important content lives. Think of it as robots.txt, but for meaning instead of permission. Ours looks roughly like this:
# AwesomeTechStack
> Website audits: technology stack, versions, quality, AI readiness.
## Key pages
- [Methodology](https://awesometechstack.com/methodology)
- [Pricing](https://awesometechstack.com/pricing)
Ten minutes of work. It is the difference between an assistant reconstructing your site from fragments and an assistant reading your own summary of it.
3. Deliberate agent access rules. AI crawlers respect robots.txt, which means robots.txt is now a business decision. Blocking GPTBot and ClaudeBot is a legitimate choice - but it should be a choice, not an accident inherited from a template. Decide who may read you, write it down:
User-agent: GPTBot
Allow: /
4. Agent identity. The emerging agent web has a discovery layer: .well-known endpoints and an agent-card that say, in machine-readable form, "this is who we are and this is what you can do here." Early days, and that is exactly the point - the sites that publish identity now are the ones agents will integrate with first.
5. Structured data. JSON-LD is the quotable layer of your site: prices, opening hours, product names, article dates, all in a form a machine can lift without guessing. Assistants strongly prefer facts they can quote over prose they must interpret. Every schema block you publish is a fact you control the wording of.
6. A real sitemap. Unglamorous, decades old, still the map every crawler follows. Present, current, and referenced from robots.txt. You would be surprised how often it's missing on otherwise professional sites - and how much of a site an agent simply never finds because of it.
Why this is worth doing now
Because the shift is already in the numbers. A growing share of product research, vendor comparison and "should I trust this company" checking happens inside assistants, and the person on the other end often never opens a browser tab at all. They ask, they get an answer, they act. If the answer was assembled without your site in it, you didn't lose the click - you lost the conversation entirely.
And because right now it's cheap. The six items above are a weekend of work for most sites, none of it requiring a rebuild. In a year, when your competitors' agent-cards are integrated and their llms.txt files are being read daily, catching up will not be a weekend.
The web has been through this before. Sites that took mobile seriously in 2010 or HTTPS seriously in 2015 didn't do anything heroic - they just moved before it was mandatory, and collected the advantage while it still was one.
Where you stand today
You don't have to guess. Run an audit on your site and the report grades AI readiness from A to F, check by check, next to your Awesomeness Score - so you can see exactly which of the six is missing and fix that one, instead of everything vaguely.
Standards like llms.txt are young and still moving; our grades are versioned, and as the AI web firms up, the checks will evolve with it. What won't change is the direction: the share of readers without browsers only grows. Build for the reader who is already in your logs.