Crawl Request Volume
The rawest discovery signal there is: how many times automated clients asked this property for something, who they claimed to be, and whether that claim holds up against published address ranges.
- Measures
- Classified crawler requests per day, grouped by vendor and by the purpose that vendor's fingerprint is assigned to.
- Data Source
- Raw server request logs, parsed and classified. First-party.
- Indicates
- That the property has been found and is being revisited. On a new domain this is the first evidence of anything at all.
- What Moves It
- New content, internal linking that exposes more of the site, sitemap freshness, server response speed, and being referenced somewhere a crawler already visits.
- Does Not Prove
- Indexing, ranking, citation, training use, or that a human ever saw the page. A request proves a request happened.
- Status
- Measured. Available from launch.
The classification step is what makes this usable rather than noisy. Unclassified, the number is dominated by scanners, scrapers, and spoofed agents, and it will happily double overnight for reasons that have nothing to do with your content.
One address presenting three or more vendors' crawler identities in a single day is a useful spoofing flag. It is a detection rule, not proof, since shared infrastructure genuinely exists.