Crawl Coverage by Section
A site-wide crawl number hides the interesting part. The useful question is which sections get requested repeatedly and which have never been fetched at all.
- Measures
- Share of published URLs in each section that received at least one classified crawler request in the period, and the revisit rate per section.
- Data Source
- Raw server request logs joined against the published URL set from the sitemap.
- Indicates
- How a crawler has prioritized the property, which is a rough read on how it has classified the sections.
- What Moves It
- Internal link structure, whether a section is reachable in few steps from the home page, sitemap inclusion, and whether the section is distinguishable from its neighbours.
- Does Not Prove
- That the fetched sections rank better, or that the unfetched ones are worse. It shows attention, not judgement.
- Status
- Measured. Needs a published URL inventory to join against, which the sitemap provides.
The most common finding from this signal is uncomfortable and cheap to fix: a section that is linked once, deep in a footer, gets crawled once. Structure is not decoration, and this is the signal that shows it plainly.