Stronger AI Safety Requires Peeking Inside the 'Black Box'
Adding security to AI often treats the large language model (LLM) or AI system as a black box, focusing on analyzing the tokenized inputs and outputs but not what goes on inside the model. The resulting techniques are often complex and individualized to specific models.
A group of offensive-security researchers hopes to change that, presenting at the Black Hat USA 2026 in August a model-agnostic approach to activation analysis with standardized rules for processing activation events. Rather than labeling certain activation distributions as "cybercrime" or "hate speech," the approach uses a more granular scheme of cognitive elements (CEs) that can be combined in rules. Combining cognitive elements such as "create content," "provide/give," "click/enter," and "personal information" in a logical statement, for example, results in a rule for detecting potential phishing attacks.
The goal is to create an open system of identified cognitive elements and rules that detect specific types of...
Copyright of this story solely belongs to darkreading.com. To see the full text click HERE
