Recall the right scraper action for common HTTP status and header scenarios.
comparison battlecard Unchanged What should a scraper do with HTTP 304 after a conditional request? Think cache validation. Treat it as a successful unchanged check, skip parsing, update checked_at, and keep the previous parsed representation. 304 is not a failed fetch. It is the server saying your stored representation is still current. Retry distinction How are 429 and 503 different for scraper action? Both can be temporary, but 429 is specifically a pacing signal. Bad advice The site returned 403. Should we rotate user agents and keep going? Access is blocked or forbidden on current terms. Your line No. Stop…
Sign up free — one personalized lesson every day, matched to your role and goals.
Already have an account? Sign in