A research paper published on ArXiv proposes a new safety module called Session Risk Memory (SRM) that detects harmful AI agent behaviour spread across multiple individually-innocent steps — a class of attack that existing safety systems are structurally unable to catch.

Current AI agent deployments typically rely on what researchers call deterministic pre-execution safety gates: rule-based checks that evaluate each action an agent takes against its assigned role and permissions. These systems work well action-by-action, but the paper identifies a fundamental gap — they carry no memory of what came before. An attacker, or a misconfigured agent, can therefore decompose a harmful goal into a sequence of steps that each look compliant in isolation, while collectively executing something prohibited.

The Attack Pattern These Systems Miss

The paper describes three specific threat scenarios its benchmark targets: slow-burn exfiltration, where sensitive data is extracted gradually over many turns; gradual privilege escalation, where an agent incrementally acquires permissions it was never meant to hold; and compliance drift, where an agent's behaviour shifts away from its authorised role over the course of a session. None of these patterns necessarily triggers a per-action safety gate, because no single step crosses a hard line.

The framework introduces a conceptual distinction between spatial authorization consistency — evaluated per action — and temporal authorization consistency — evaluated over trajectory.

SRM addresses this by maintaining what the authors call a semantic centroid: a compact running summary of an agent session's behavioural profile, updated continuously as the session progresses. Rather than adding a new machine learning model or requiring additional training, SRM operates directly on the same semantic vector representations already used by the underlying safety gate. This makes it a modular addition rather than an architectural overhaul.

How the Risk Signal Accumulates

The technical mechanism works through an exponential moving average applied to risk signals generated by the existing gate at each step. Crucially, each signal is baseline-subtracted — meaning SRM is sensitive to deviation from an agent's own established behavioural norm, not just deviation from a static threshold. Over time, a session that is drifting toward harmful territory accumulates a rising risk score, even if each individual step scores as low-risk in isolation.

The authors tested SRM on a benchmark of 80 multi-turn sessions, purpose-built to include the three attack categories described above. The results they report — which are from the authors' own evaluation and have not yet undergone independent peer review — show that combining SRM with the existing ILION safety gate achieves an F1 score of 1.0000 with a 0% false positive rate. The stateless version of ILION alone scored an F1 of 0.9756 with a 5% false positive rate. Both systems maintained a 100% detection rate for genuinely malicious sessions.

The False Positive Problem Matters as Much as Detection

The false positive result deserves particular attention. In deployed agentic systems — AI agents that autonomously execute tasks, make API calls, or interact with external services — false positives are not merely an inconvenience. Each false positive means a legitimate agent action is being blocked, which can interrupt workflows, frustrate users, and create pressure on operators to loosen safety thresholds. A system that eliminates false positives while maintaining perfect detection offers a more practical path to real-world deployment than one that trades off between the two.

The computational cost SRM adds is reported at under 250 microseconds per turn, which the authors characterise as lightweight. For context, agentic AI systems typically operate at timescales measured in seconds per action, meaning SRM's overhead would be imperceptible in practice.

Where This Fits in the Broader Safety Landscape

The paper frames its contribution partly as a conceptual one: it formally distinguishes between spatial authorization consistency (is this action compatible with this agent's role, right now?) and temporal authorization consistency (is this session's overall trajectory compatible with this agent's role over time?). This distinction has not been explicitly drawn in prior work on agentic safety gates, according to the authors.

The research arrives at a moment when multi-agent and autonomous AI systems are moving from research prototypes toward production deployments across industries including software development, customer operations, and scientific research. As these systems are given more capability to take consequential actions — executing code, accessing databases, sending communications — the attack surface for trajectory-level manipulation grows correspondingly. A single-step safety gate was designed for a narrower threat model.

The paper does not claim SRM is a complete safety solution. It operates within the same semantic framework as the underlying gate, which means it inherits any limitations of that gate's representation. Sessions that evade the base gate's semantic detection would likely evade SRM as well. The system is designed to add a layer, not to replace existing mechanisms.

What This Means

If the results hold under independent evaluation, SRM offers AI developers a low-cost, no-training upgrade to existing safety infrastructure that closes a documented structural gap — making it harder to manipulate agentic systems through gradual, distributed attacks that no single checkpoint would catch.