A Methodology for Quantitative AI Risk Modeling

Although general-purpose AI systems offer transformational opportunities in science and industry, they simultaneously raise critical concerns about safety, misuse, and potential loss of control. Despite these risks, methods for assessing and managing them remain underdeveloped. Effective risk management requires systematic modeling to characterize potential harms, as emphasized in frameworks such as the EU General-Purpose AI Code of Practice. This paper advances the risk modeling component of AI risk management by introducing a methodology that integrates scenario building with quantitative risk estimation, drawing on established approaches from other high-risk industries. Our methodology models risks through a six-step process: (1) defining risk scenarios, (2) decomposing them into quantifiable parameters, (3) quantifying baseline risk without AI models, (4) identifying key risk indicators such as benchmarks, (5) mapping these indicators to model parameters to estimate LLM uplift, and (6) aggregating individual parameters into risk estimates that enable concrete claims (e.g., X% probability of >$Y in annual cyber damages). We examine the choices that underlie our methodology throughout the article, with discussions of strengths, limitations, and implications for future research. Our methodology is designed to be applicable to key systemic AI risks, including cyber offense, biological weapon development, harmful manipulation, and loss-of-control, and is validated through extensive application in LLM-enabled cyber offense. Detailed empirical results and cyber-specific insights are presented in a companion paper.

💡 Research Summary

The paper addresses a critical gap in current AI risk management: while many frontier AI companies and regulatory frameworks focus on measuring model capabilities (e.g., benchmark scores, capability thresholds), they do not directly quantify the actual risks that those capabilities may generate. To bridge this gap, the authors propose a comprehensive, six‑step methodology for quantitative AI risk modeling that can be applied to the four systemic risk categories identified in the EU AI Act’s Code of Practice—cyber offense, biological weapon development, harmful manipulation, and loss of control.

Step 1 – Define risk scenarios
The process begins by enumerating plausible, high‑impact pathways from a hazardous AI capability to real‑world harm. The authors combine top‑down Fault Tree Analysis (starting from an undesired outcome) with bottom‑up Event Tree Analysis (starting from an initiating event) to map causal chains. Each scenario becomes a distinct risk model.

Step 2 – Decompose scenarios into measurable parameters
Every scenario is broken into three core components that together constitute the risk equation: (a) the frequency with which the initiating event chain is triggered, (b) the probability that the entire chain will successfully unfold, and (c) the magnitude of the resulting damage. This mirrors established risk‑assessment practices in aviation, nuclear, and other safety‑critical domains.

Step 3 – Quantify baseline risk without AI
Before adding any AI‑specific uplift, the methodology establishes a “baseline” risk using historical data, industry reports, or expert‑derived estimates for the same threat vector absent AI assistance. This baseline serves as a reference point for measuring the incremental effect of AI.

Step 4 – Identify key risk indicators (KRIs)
The authors collect a basket of observable indicators—benchmark scores, red‑team exercise outcomes, incident reports, and expert judgments—that can be linked to the parameters defined in Step 2. These KRIs act as proxies for the underlying probabilities and frequencies.

Step 5 – Map KRIs to model parameters to estimate LLM uplift
Using a modified Delphi process (the IDEA protocol) and, where possible, empirical uplift studies, the methodology translates each KRI into a quantitative “uplift factor” that captures how a large language model (LLM) changes the parameter values. For example, an LLM might reduce the cost of a reconnaissance step by 30 % and double the success probability of a penetration step.

Step 6 – Aggregate and propagate uncertainty
The final step integrates the uplift‑adjusted parameters into a probabilistic model. The authors employ Bayesian networks to capture causal dependencies among parameters and Monte Carlo simulation to propagate uncertainty, producing full probability distributions and confidence intervals for the overall risk. This enables concrete claims such as “there is a 5 % probability that AI‑enabled cyber attacks will cause annual damages exceeding $100 million.”

The methodology is demonstrated on the domain of AI‑enabled cyber offense. In that case study, expert elicitation indicated that LLMs cut the time required to generate phishing templates by 40 % and increased the likelihood of successful credential harvesting by a factor of 1.8. When combined with baseline cyber‑attack statistics, the model predicts an increase in expected annual loss from $200 M to $350 M and a rise in the probability of losses above $100 M from 7 % to 12 %. The authors argue that such quantitative statements are directly comparable to regulatory thresholds used in other high‑risk sectors (e.g., the FAA’s one‑per‑billion‑flight‑hours catastrophic‑event limit).

Strengths

1. Transparency – By separating scenario definition, parameter decomposition, baseline estimation, and AI uplift, the approach makes each modeling assumption explicit.
2. Hybrid evidence – The combination of expert elicitation and data‑driven estimates mitigates the scarcity of historical AI‑related incident data.
3. Uncertainty quantification – Bayesian networks and Monte Carlo simulation provide probabilistic outputs rather than single point estimates, supporting risk‑based decision making.
4. Policy relevance – The resulting risk metrics can be directly compared with existing safety standards, facilitating the creation of AI‑specific risk thresholds for regulators and industry.

Limitations

• Scenario selection bias – The set of scenarios is necessarily curated; missing a plausible pathway could underestimate risk.
• Independence assumptions – While Bayesian networks model dependencies, many parameter relationships are still approximated as independent, which may understate joint risk.
• Domain specificity – The empirical validation is limited to cyber offense; applying the same framework to biological weaponization, CBRN, or loss‑of‑control will require domain‑specific data and possibly different KRIs.
• Data paucity – Real‑world AI‑driven incidents are rare, so model validation relies heavily on expert judgment, which can introduce systematic bias.

Future research directions suggested by the authors include: (i) building continuous feedback loops with incident databases to update model parameters, (ii) extending the framework to capture multi‑risk interactions (e.g., a cyber breach that enables the synthesis of a pathogen), (iii) developing standardized risk‑thresholds for AI comparable to those in aviation or nuclear safety, and (iv) conducting cross‑domain pilots to test the methodology’s generality.

In sum, the paper delivers a rigorous, reproducible, and policy‑oriented methodology for turning vague concerns about “AI risk” into concrete, quantitative risk estimates. By doing so, it equips regulators, AI developers, insurers, and corporate risk officers with a tool to prioritize mitigations, allocate resources, and ultimately reduce the probability and impact of AI‑enabled harms.