The Case for Context-Aware Model Evaluation
Every model review I have sat in eventually arrives at the same slide: one big number in a large font. Accuracy 92%. F1 0.88. AUC 0.91. Heads nod. Someone asks about latency—the model ships.
The problem is not that the number is wrong. The problem is that it answers a question nobody actually asked. "How often is the model right, averaged over everyone in the test set?" is not the same as "Is this model safe to make decisions about the people it will be used on?"
Those two questions come apart whenever the population is not uniform, which is to say, always. Your users are not one distribution. They are several distributions stacked on top of each other, and the biggest one dominates the average.
Accuracy tells you how good the model is for the average user but nobody is the average user.
A concrete example: the thin-file problem
...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
