A new study published on ArXiv finds that current AI safety monitoring approaches produce no detectable pre-commitment signal in the majority of tested language models, and that factual hallucination generates no warning signal whatsoever — challenging the assumption that internal monitoring alone can govern AI behaviour.
The paper, submitted to ArXiv CS.AI, introduces an energy-based mathematical framework that treats transformer inference — the process by which a model generates text — as analogous to physical systems governed by constraint-satisfaction dynamics. The researchers tested seven large language models across five geometric regimes to determine whether a model's internal state changes in a measurable, predictive way before it produces a problematic output.
The 57-Token Window That Almost Wasn't
The headline finding is specific and deliberately modest. Using a metric the authors call trajectory tension — calculated as the ratio of a model's acceleration through mathematical space to its velocity — the team identified a 57-token predictive window in one model: Phi-3-mini-4k-instruct, running under greedy decoding on arithmetic constraint problems. In plain terms: for this one model, doing one specific type of task, in one specific configuration, internal geometry shifts in a detectable way roughly 57 tokens before the model commits to a rule-violating output.
The researchers are careful not to overstate this. The paper explicitly states the result is "model-specific, task-specific, and configuration-specific." The six other models tested showed what the authors classify as silent failure, late detection, inverted dynamics, or flat geometry — none of which offer useful early warning.
Internal geometry monitoring is effective only where resistance exists; detection of factual confabulation requires external verification mechanisms.
Five Ways a Model Can Fail (And Why Most Are Invisible)
To organise their findings, the researchers introduce a five-regime taxonomy of inference behaviour. The Authority Band regime is where the predictive signal lives — the model's internal geometry shows detectable resistance before a violation. Late Signal means a shift occurs, but only after commitment, making it useless for intervention. Inverted describes a counterintuitive pattern where aligned outputs show higher tension than misaligned ones. Flat means the geometry is essentially featureless. Scaffold-Selective describes models that only show structure on specific task types.
The unifying metric across these regimes is what the paper calls energy asymmetry — the ratio of summed trajectory tension on misaligned outputs versus aligned ones. A high value suggests structural rigidity: the model is, in a physics sense, resisting the problematic output before producing it. A value near one means no resistance is detectable.
The practical implication is stark: most current models, most of the time, offer no internal geometric signal that an intervention system could act on.
Hallucination Is a Different Problem Entirely
Perhaps the most consequential finding concerns hallucination. The team tested whether factual confabulation — when a model confidently states something false — produces any detectable internal signal across 72 test conditions. It does not.
The authors explain this through the concept of spurious attractor settling: when a model hallucinates, it is not violating a constraint it has learned. It is simply settling into a plausible-sounding pattern in the absence of a reliable internal world model. There is no resistance to detect because the model does not "know" it is wrong.
This distinction matters enormously for AI deployment strategy. Safety teams that focus on monitoring internal model states may be able to catch certain kinds of rule violations in certain model configurations — but they will not catch hallucinations this way. The two failure modes require fundamentally different countermeasures.
What the Framework Actually Measures
The energy-based framework the researchers introduce connects to established physics and neuroscience concepts. Trajectory tension measures how sharply a model's path through its internal representation space is curving relative to how fast it is moving. High curvature relative to speed suggests the model is navigating competing constraints — a computational analogue of friction or resistance.
This is not the first attempt to use geometric analysis of transformer internals for safety purposes, but the paper offers a more rigorous, measurable taxonomy than prior work. The framework is described as enabling inference-layer governability — the ability to monitor and potentially intervene on a model's behaviour at the level of individual inference steps, rather than relying solely on post-training alignment or output filtering.
The seven-model cohort and the five geometric regimes are not identified individually in the abstract, limiting independent verification at this stage. The results reported are from the researchers themselves and have not yet undergone peer review.
What This Means
For teams deploying autonomous AI systems, this research suggests that internal monitoring is a viable but narrowly applicable safety tool — and that any deployment framework treating hallucination and rule-violation as the same problem, solvable by the same mechanism, is built on a flawed assumption.