Instrument cache hits in one afternoon
Create a minimal cache-hit instrumentation plan with prompt version, token fields, prefix hash, and latency.
A document summarizer was refactored for caching, but the team only has total weekly spend. They need proof before rollout. DMAIC measurement: define the process, capture trustworthy baseline data, then improve one variable at a time. The weak move is to use invoice change as proof. Cost can move because traffic, output length, retries, or model mix changed. Step 1 Add identifiers: workflow=document_summary, prompt_version=summary_v7, model, provider. You need a stable grouping key before comparing anything. Step 2 Capture token fields: input_tokens, cached_tokens, output_tokens. Cached input tokens are the direct signal that the prompt shape is getting reuse. Step 3 Capture…
Sign up free — one personalized lesson every day, matched to your role and goals.
Already have an account? Sign in