Bucket rare categories before encoding

Prepare device_model for a signup-fraud classifier with thousands of rare values and new devices every release. Fit category support thresholds on training data, pool rare values, and preserve a deterministic fallback for unseen values. One-hot encoding the full tail can memorize rare labels, while recomputing thresholds after splitting leaks validation support. Profile Count device_model support on the training fold only. Validation and test rows cannot help decide which categories are rare. Threshold Map categories with fewer than 100 training observations to __RARE_DEVICE__. The threshold converts noisy one-row categories into a generalizable state. Fallback Map production categories not seen in training…

Sign up free — one personalized lesson every day, matched to your role and goals.

Already have an account? Sign in