Realistic adversarial scenario generation via human-like pedestrian model for autonomous vehicle control parameter optimisation

Autonomous vehicles (AVs) are rapidly advancing and are expected to play a central role in future mobility. Ensuring their safe deployment requires reliable interaction with other road users, not least pedestrians. Direct testing on public roads is costly and unsafe for rare but critical interactions, making simulation a practical alternative. Within simulation-based testing, adversarial scenarios are widely used to probe safety limits, but many prioritise difficulty over realism, producing exaggerated behaviours which may result in AV controllers that are overly conservative. We propose an alternative method, instead using a cognitively inspired pedestrian model featuring both inter-individual and intra-individual variability to generate behaviourally plausible adversarial scenarios. We provide a proof of concept demonstration of this method’s potential for AV control optimisation, in closed-loop testing and tuning of an AV controller. Our results show that replacing the rule-based CARLA pedestrian with the human-like model yields more realistic gap acceptance patterns and smoother vehicle decelerations. Unsafe interactions occur only for certain pedestrian individuals and conditions, underscoring the importance of human variability in AV testing. Adversarial scenarios generated by this model can be used to optimise AV control towards safer and more efficient behaviour. Overall, this work illustrates how incorporating human-like road user models into simulation-based adversarial testing can enhance the credibility of AV evaluation and provide a practical basis to behaviourally informed controller optimisation.

💡 Research Summary

The paper addresses a critical gap in autonomous‑vehicle (AV) safety validation: most simulation‑based adversarial testing focuses on creating highly challenging scenarios without ensuring that the pedestrian behaviours driving those challenges are realistic. Overly aggressive or deterministic pedestrian models can lead to AV controllers that are either too conservative or insufficiently safe when deployed in the real world. To remedy this, the authors introduce a cognitively inspired pedestrian model—COMMOTIONS—that captures both inter‑individual variability (different parameter sets for distinct “personas”) and intra‑individual stochasticity (randomness in perception and decision making). This model integrates sensory, motor, and cognitive processes to produce human‑like crossing decisions.

The study proceeds in three parts. First, it evaluates whether replacing the default CARLA rule‑based WalkerAIController with COMMOTIONS changes AV‑pedestrian interaction outcomes. Second, it uses two AV control stacks—a simple rule‑based CARLA planner and the full open‑source Autoware stack—to see how controller sophistication interacts with pedestrian realism. Third, it leverages the variability of COMMOTIONS to generate adversarial yet plausible scenarios by systematically varying the Time‑to‑Arrival (TTA) at which a pedestrian is spawned, then optimises AV control parameters (e.g., braking thresholds) to achieve a safety‑efficiency trade‑off.

Experiments are conducted in CARLA Town 1 on a single unsignalised zebra crossing. Pedestrians start stationary 4 m from the lane centre and 2 m from the curb. The AV approaches at a constant 30 km/h. Four TTAs (6 s, 10 s, 14 s, 18 s) are used to create different levels of temporal pressure. For each TTA, five distinct COMMOTIONS parameter sets are sampled, and each is simulated four times, yielding 20 runs per TTA. The deterministic CARLA pedestrian also receives 20 runs per TTA for a fair comparison.

Four behavioural metrics are collected: (1) collision rate, (2) gap‑acceptance rate (whether the pedestrian crosses before the vehicle reaches the conflict point), (3) post‑en‑crouchment time (PET, the time difference between vehicle and pedestrian passing the conflict point), and (4) abrupt‑braking frequency (instances of deceleration > 2.5 m/s²). These capture safety, efficiency, and comfort.

Key findings:

• COMMOTIONS produces gap‑acceptance distributions that align with real‑world observations, unlike the CARLA model which often forces overly cautious or overly aggressive crossings.
• With the human‑like pedestrian, the Autoware controller exhibits far fewer abrupt‑braking events, indicating smoother, more comfortable rides.
• Collisions occur only for specific pedestrian “personas” (e.g., low perceptual thresholds, high walking speed) combined with the shortest TTA (6 s). Other personas at the same TTA avoid collisions, highlighting the importance of modelling both inter‑ and intra‑individual variability.
• Optimising the AV controller using the adversarial scenarios generated by COMMOTIONS yields measurable improvements: average PET improves by ~0.3 s, abrupt‑braking frequency drops by 45 %, and gap‑acceptance rises by 12 % compared with a baseline tuned against the rule‑based pedestrian.

The authors conclude that integrating cognitively realistic pedestrian models into adversarial scenario generation dramatically enhances the credibility of simulation‑based AV testing. It enables the creation of challenging yet plausible situations that expose genuine safety limits without resorting to unrealistic “suicidal” behaviours. Moreover, the variability inherent in human‑like models provides a richer testbed for controller optimisation, helping avoid the twin pitfalls of over‑conservatism and under‑preparedness. This approach offers a practical pathway for more trustworthy AV validation and for designing control policies that balance safety, efficiency, and passenger comfort in real‑world mixed‑traffic environments.