Skip to the content
searchboar.

Crawling and indexing

HTTP 429: Meaning, examples and limitations

HTTP 429 Too Many Requests means that a client has exceeded a request rate accepted by a server or intermediary. It asks the requester to slow down; it does not mean the requested page has been permanently removed.

At a glance

  • 429 communicates rate limiting, not a missing page.
  • Retry-After can indicate when another attempt is appropriate.
  • Broad rate limits can affect legitimate visitors and crawlers too.
On this page
Request cards queue at a gate marked with a pause symbol.
HTTP 429 Too Many Requests means that a client has exceeded a request rate accepted by a server or intermediary.

Where does a rate limit apply?

A limit may be associated with an IP address, account, endpoint or another rule chosen by the service. A CDN or firewall can return 429 before the application handles the request. The status alone does not identify which layer enforced the limit.

A response can include Retry-After to indicate a waiting period or retry time. Clients should respect the service's retry guidance rather than immediately repeat the same request. HTTP reference

Example: repeated directory requests

Imagine a public directory receiving hundreds of rapid requests from one client. A rate limit may temporarily reject that client's requests while ordinary readers continue browsing. But a rule based only on a shared IP can also affect multiple unrelated people behind the same network.

Before changing the rule, compare the requested paths, timing and response source. A legitimate search crawler and an abusive scraper can both send many requests; volume alone does not prove identity. Bot verification is a separate check.

What is the SEO consequence?

If useful pages repeatedly return 429 to a crawler, it cannot reliably retrieve their content. A short-lived limit and persistent exclusion are different situations. Review server logs and crawl behavior rather than treating the existence of a firewall rule as proof that search traffic is unaffected.

Do not present a throttling page as the normal content of a URL. The actual status should match the refusal. A permanent missing page needs another response; 404 does not mean “try again more slowly.”

Does it guarantee a small hosting bill?

No. Requests may consume resources before a limit runs, and different hosting products charge for different activities. Caching public responses, limiting expensive operations and observing real usage are separate controls. A 429 policy is useful only when its scope, placement and effect are understood; it is not a universal cost cap.

Google describes crawler-specific responses to these status codes in its HTTP status reference.

Sources

  1. MDN: HTTP 429
  2. Google: HTTP status codes and crawler behavior