Researchers have developed an automated framework that uses large language models to generate safety-critical fault scenarios for autonomous driving systems, uncovering significant performance failures that standard testing methods routinely miss.

The study, published on ArXiv in April 2025, targets a specific and growing problem: autonomous vision systems deployed on resource-constrained edge devices — think embedded chips in vehicles or roadside units — cannot run heavy safety-validation software in real time. Current industry practice relies on static datasets or manually designed fault injections, neither of which adequately reflects the chaotic variety of conditions a vehicle encounters on actual roads.

How the Framework Separates Heavy Computation from Real-Time Testing

The proposed system, which the authors call a decoupled offline-online fault injection framework, splits validation into two distinct phases. In the offline phase, an LLM semantically generates structured descriptions of fault scenarios — fog, sensor glare, camera blur, rain — and a Latent Diffusion Model (LDM) then synthesises photorealistic versions of those degraded sensor inputs. The outputs are compressed into a pre-computed lookup table.

In the online phase, the edge device consults that lookup table during inference, gaining fault-awareness without needing to run any heavy AI models locally. The architecture is designed so that the computationally expensive work happens once, centrally, and the lightweight result is what gets deployed to hardware with limited processing power.

Our generated faults expose significant robustness degradation, with RMSE increasing by up to 99% and within-0.10 localization accuracy dropping to as low as 31.0% under fog conditions.

This separation addresses a genuine engineering constraint. Edge chips used in automotive and robotics applications are often chosen for power efficiency, not raw compute. Running a diffusion model or an LLM inference loop on such hardware in real time is not currently feasible at scale.

A Lane-Following Model Fails Under Fog — Badly

The researchers validated their framework against a ResNet18 lane-following model — a well-established convolutional neural network architecture commonly used as a benchmark in embedded vision tasks — across 460 generated fault scenarios. On clean data, the model performed solidly, achieving a baseline R² score of approximately 0.85, meaning it explained about 85% of the variance in steering predictions.

Under generated fault conditions, performance degraded significantly. Root Mean Square Error (RMSE) — a measure of how far the model's lane predictions deviated from ground truth — increased by up to 99%. More strikingly, the proportion of predictions falling within an acceptable localization tolerance of 0.10 units dropped to 31.0% under simulated fog. In practical terms, the model was inaccurate about where the lane was nearly seven times out of ten under a single common weather condition.

These figures are self-reported by the research team and have not been independently replicated.

Why Static Datasets Miss the Problem

The core argument the paper makes is methodological: evaluating AI systems only on clean or pre-curated data produces an overly optimistic picture of real-world reliability. Static datasets, by definition, cannot cover every environmental condition a deployed system will encounter. Manual fault injection is labour-intensive and inherently limited by the imagination of the engineers designing the tests.

LLMs offer a different approach. Because they are trained on broad corpora of text describing the physical world, they can be prompted to enumerate plausible fault types systematically — generating a far wider catalogue of degradation scenarios than a human team might produce manually. The diffusion model then translates those text descriptions into synthetic images that realistically simulate how a camera sensor would respond to each condition.

The combination amounts to an automated stress-testing pipeline. Rather than asking engineers to anticipate every possible failure mode, the system generates them computationally.

Implications for Edge AI Safety Standards

The autonomous vehicle and robotics industries are under increasing regulatory pressure to demonstrate that AI systems meet safety standards before deployment. Bodies including the ISO (with its ISO 26262 functional safety standard for automotive systems) and various national transport authorities are developing requirements around AI validation. The challenge is that existing standards were largely written with traditional software in mind, not probabilistic neural networks.

Frameworks like this one could feed into a more scalable approach to certification. If an LLM-driven system can automatically generate thousands of diverse fault scenarios, safety engineers could run broader validation sweeps at lower cost. The lookup-table architecture also means that fault-aware behaviour could be embedded into already-certified hardware without requiring a full re-certification of the compute stack.

However, the approach carries its own limitations. LLMs generate scenarios based on patterns in training data, which means genuinely novel or rare failure modes — the kind most likely to cause accidents in practice — may still be underrepresented. The quality of the synthetic images produced by the diffusion model also depends heavily on how well those models have been trained on sensor-degradation data specifically.

What This Means

For engineers and safety teams deploying AI on edge hardware, this research underscores a practical concern: a model that performs well on clean benchmarks may be substantially unreliable in common real-world conditions, and the only way to know is to test against a far wider range of fault scenarios than current practice typically demands.