We Used Benchmaxxers' Favourite Trick to Climb 10 Places on the Hugging Face Open ASR Leaderboard
TL;DR: There is a well-known trick for climbing ASR leaderboards: fine-tune on data that closely resembles a testset, submit, watch your rank improve. We did exactly that with AMI, the one testset that consistently separates serious models from optimised ones. We went from rank 26 to rank 16. Here is what actually happened, what broke along the way, and what it means for how you read anyone's leaderboard score.
Everyone knows leaderboards get gamed
Public benchmarks have a problem. The moment a testset becomes the standard measure of model quality, it also becomes the optimisation target. Teams fine-tune on training data that resembles the testset, run inference configurations tuned specifically for evaluation conditions, and submit. The number goes up. The leaderboard moves. Whether the model has actually improved at the underlying task is a separate question.
This is not a secret. It is just rarely talked about openly, because talking...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
