Sort HTTP outcomes into scraper actions

Place each response scenario in the action bucket a responsible scraper should use. Parse or store Redirect or unchanged Skip or retire Backoff or retry later Escalate access HTTP 200 with expected HTML and all required fields present HTTP 200 with expected JSON endpoint and valid schema HTTP 301 from old product URL to canonical product URL HTTP 304 after sending If-None-Match with yesterday's ETag HTTP 404 for a product URL that no longer appears in the sitemap HTTP 429 with Retry-After header from the source host HTTP 503 during a short outage window on an otherwise allowed page HTTP…

Sign up free — one personalized lesson every day, matched to your role and goals.

Already have an account? Sign in