Grok chat duped into swallowing injected instructions
ai and ml
A spoonful of encryption helps the malware go down
xAI's Grok web chat agent is currently vulnerable to a novel form of prompt injection, according to security researchers with Adversa AI.
The technique allows an attacker to create a web page poisoned with malicious instructions that induce an AI model summarizing the page to carry out harmful actions.
That describes a well-known attack known as indirect prompt injection. Frontier AI models have become better at dealing with such attempts through existing guardrails, though the issue is far from resolved.
Adversa's approach comes with a twist: It relies on encrypted malicious instructions, which attackers place on a web page alongside an encryption key. The model guardrail scanner – an input filter – can't read the encrypted text despite the presence of the key. The scanner therefore passes it on to the model, which can use the key...
Copyright of this story solely belongs to theregister.com. To see the full text click HERE
