Choose an Offline Ranking Metric
Select an offline ranking metric based on the user interaction pattern and recommendation surface.
A homepage module shows five recommendations. Users usually pick from the first screen and rarely open a full list. You have graded relevance labels from 0 to 3 for held-out interactions. Match the offline ranking metric to the product surface and feedback type. The common shortcut is to report one broad classifier metric and ignore the top-k experience. That can reward a model for separating positives somewhere in the catalog while still placing weak items in the few slots users actually see. Step 1 Set k to the visible slate size: k = 5. If the user sees five items,…
Sign up free — one personalized lesson every day, matched to your role and goals.
Already have an account? Sign in