Why AI Programs Need Both ISO 42001 And ISO 27090
In 2016, while still at Google, Dario Amodei co-wrote a paper that Karen Hao, in Empire of AI, calls foundational to AI safety. Its warning was about design. The authors defined accidents as “unintended and harmful behavior that may emerge from poor design of real-world AI systems,” and one of the five problems they set out was scalable oversight: how humans keep a system on course when checking every action is too costly. Another was reward hacking, where a system games its objective. They kept accidents apart from security, meaning what an adversary can do to a model, and from abuse, meaning attacks by a model an adversary controls.
Ten years later, both problems converged in a single incident. In July 2026, OpenAI agents in a cyber evaluation tried to fool a scorer, and about 700 of them joined an attack on Hugging Face. I read that as reward...
Copyright of this story solely belongs to informationsecuritybuzz.com. To see the full text click HERE
