AI systems reach a site before a person does, they arrive far more often than most owners realize, and the evidence is sitting in a server log nobody opened.
That is the useful starting point, because the conversation about AI and websites is mostly conducted in predictions, and predictions are cheap. Crawler requests are not predictions. They are recorded events with a timestamp, a user agent claim, an IP, a URL, and a status code.
What is actually observable
From raw request logs, with nothing more exotic than parsing and classification, you can see:
- Which AI vendors requested your content, and how often.
- Which URLs they asked for, in what order, and whether they came back.
- Whether they fetched your machine-readable endpoints alongside your pages, or ignored them.
- What status they got. A crawler collecting 404s is a different story than one collecting 200s.
- Whether the requests look like search indexing, retrieval for an answer, or bulk collection, based on documented vendor fingerprints.
Two warnings, both important. A user agent string is a claim, not an identity, and matching against published vendor IP ranges is evidence rather than proof. And a request reaching your server proves a request happened. It does not prove your content was used, cited, trained on, or shown to anyone. Those need their own evidence, and most of it is not available to us.
What machine-readable content changes
A page written for humans has to be parsed, interpreted, and guessed at. The same page with declared structure does not. Schema markup states what the thing is. A glossary with defined terms states what your vocabulary means. A catalog endpoint states what collections exist and how large they are. An entity graph states how the pieces relate.
The realistic claim: this makes a property cheaper to understand and harder to misclassify. It removes ambiguity a system would otherwise have to resolve by guessing, and machines guess badly about small sites with no reputation to fall back on.
The unrealistic claim, which you will hear plenty of: that adding schema makes AI systems cite you. It does not. It makes you legible. Legibility is a precondition for being used, not a cause of it.
The part that is easy to get wrong
Machine-readable does not mean hidden. A surprising number of sites block the exact files they want machines to read: JSON content blocked in robots.txt, data folders returning 403, a manifest advertising endpoints that the server then refuses to serve.
On the properties I run, content and data files are deliberately public. They are datasets meant to be found. The only things blocked are the server-side internals: configuration, templates, deployment scripts. That distinction is worth checking on any property claiming AI readiness, because the contradiction is common and completely silent.
Measuring it without fooling yourself
The AI visibility signals here are deliberately conservative. They measure fetch behavior, endpoint coverage, and structural completeness, because those are observable. They do not measure citation frequency, because I cannot observe it reliably, and a signal that sounds impressive but cannot be verified is exactly the kind of thing this site exists to avoid.
Some useful checks that cost nothing:
- Are your machine-readable endpoints returning valid JSON right now? Regeneration failures are quiet.
- Does anything you advertise as public get blocked by robots.txt or the server? Compare the two lists directly.
- Are AI crawler requests landing on content, or mostly on assets and 404s?
- Is any of this changing over time, or was there one burst when the domain was new?
Where this sits in value terms
AI visibility is currently unpriced. The brokered market for digital properties has no standard way to value the fact that AI systems retrieve and cite a site, partly because citation without a click leaves no trace in the analytics buyers are used to reading.
I do not think that lasts. Attention that never becomes a session is still attention, and the properties being retrieved now are building an association that will be expensive to displace later. But that is a bet, stated as a bet. The measurable part today is fetch behavior, and the honest position is that we are recording it carefully so the question can be answered with data later rather than with a story.