Skip to content

Crawl Budget

The volume of fetching a crawler will spend on a site in a period.

Budget is spent before it is earned. Slow responses, redirect chains, near-duplicate pages and parameter sprawl consume fetches that could have gone to pages that matter. For large travel sites with faceted inventory this is often the difference between being current in the retrieval layer and being a year out of date.

Related terms

Related indicators