Nobody Reviewed the Model. They Just Reviewed the Code Around It
Before signing a contract, a customer's security team conducted a vendor review, and one question on their list left our ML lead speechless: “Which exact model weights are running in production, and who reviewed the code that loads them?” We answered the first half. We knew the model's name on Hugging Face. We had no idea which revision we were actually running, because our Dockerfile pulled “latest” from that repo every time we rebuilt the image. And the second half of the question was worse: our loading code passed trust_remote_code=True, which meant a Python file written by a stranger, hosted on someone else's repo, ran automatically inside our container every time it started. Nobody on the team could tell the auditor what was in that file, because nobody had ever opened it.
That conversation didn't end the deal, but it should have ended a lot sooner, with us catching it...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
