Researcher poisons open-weight AI model for under $100
AI and ML
Models demand trust without offering verification
The AI supply chain is, in some ways, even more vulnerable to poisoning than that of traditional software.
Katie Paxton-Fear, a lecturer in cybersecurity at Manchester Metropolitan University and staff security advocate at Semgrep, managed to install a backdoor in an open-weight AI model in about an hour for less than $100.
"I started out by trying to figure out if I could use fine tuning to get a model to swap from camelCase for JavaScript to snake_case, and it was actually really easy, even if we then gave the AI specific instructions to use camelCase," Paxton-Fear wrote in a recent social media post. "After that worked, I did a proper backdoor."
It only took ten training examples for the code output by the model to become reliably vulnerable to remote code execution, even for novel prompts and domains, she...
Copyright of this story solely belongs to theregister.com. To see the full text click HERE
