AEGIS: White-Box Attack Path Generation using LLMs and Training Effectiveness Evaluation for Large-Scale Cyber Defence Exercises

Creating attack paths for cyber defence exercises requires substantial expert effort. Existing automation requires vulnerability graphs or exploit sets curated in advance, limiting where it can be applied. We present AEGIS, a system that generates attack paths using LLMs, white-box access, and Monte Carlo Tree Search over real exploit execution. LLM-based search discovers exploits dynamically without pre-existing vulnerability graphs, while white-box access enables validating exploits in isolation before committing to attack paths. Evaluation at CIDeX 2025, a large-scale exercise spanning 46 IT hosts, showed that AEGIS-generated paths are comparable to human-authored scenarios across four dimensions of training experience (perceived learning, engagement, believability, challenge). Results were measured with a validated questionnaire extensible to general simulation-based training. By automating exploit chain discovery and validation, AEGIS reduces scenario development from months to days, shifting expert effort from technical validation to scenario design.

💡 Research Summary

The paper introduces AEGIS, a novel system that automates the creation of attack paths for large‑scale cyber‑defence exercises (CDX) by combining large language models (LLMs), white‑box access to the target network, and Monte‑Carlo Tree Search (MCTS) over real exploit execution. Traditional automation tools require pre‑constructed vulnerability graphs or curated exploit libraries, which limits their applicability and still demands extensive expert effort to validate and stitch together realistic attack narratives. AEGIS addresses these shortcomings through a staged pipeline that decouples vulnerability discovery from exploit execution.

The pipeline consists of four main components. First, a Reconnaissance Scanner uses PowerCLI credentials to obtain direct access to every virtual machine in a VMware vSphere environment. It runs host‑level package and service scans, then performs network‑level port scans to build a directed graph of reachable ports. LLM agents are only used to select appropriate scan commands; the actual scanning is performed by scripted tools, mirroring how human red‑teamers use playbooks.

Second, an Exploit Searcher takes each (software, version) pair identified by the scanner and queries an LLM‑driven internet search API (Tavily Search) for all associated CVE identifiers. Rather than relying on structured results, the system extracts every CVE number from raw text and then filters out irrelevant entries using a second LLM prompt that explicitly asks for exclusions. For each remaining CVE, the system crawls GitHub and Exploit‑DB, evaluates the quality and completeness of the exploit code with another LLM (assigning a 1‑4 score for execution difficulty and documentation quality), and prioritises exploits with the highest combined score.

Third, the Exploit Validator empirically tests each candidate exploit in isolation. Validation is performed from a Kali Linux host placed in the same network compartment as the target, ensuring that reachability does not confound exploitability. For initial‑access exploits, an unprivileged SSH session is provisioned to simulate a foothold; for privilege‑escalation exploits, the target is accessed directly. Success is confirmed by the creation of a known file or the launch of a specific process, providing a concrete, machine‑readable signal that the exploit works. This white‑box validation eliminates the “assume‑vs‑test” error observed in prior LLM‑based pentesting tools.

Finally, the Attack Path Simulator employs MCTS where each action corresponds to a real exploit execution rather than a simulated step. Because real exploits take minutes to run, the action space is dramatically reduced by (a) discarding exploits that failed validation and (b) ordering remaining exploits according to the LLM‑derived priority scores. The MCTS algorithm expands the tree by executing the selected exploit, observes the resulting system state, and continues until a user‑specified goal state (e.g., compromise of a critical server) is reached. Operators provide the goal and any post‑exploitation objectives; AEGIS returns a complete, executable attack chain.

The system was evaluated during CIDeX 2025, a large‑scale cyber‑defence exercise involving 46 hosts (Linux and Windows servers, client workstations, and network appliances). The authors compared two human‑crafted attack paths with those generated automatically by AEGIS. Training effectiveness was measured using a 12‑item questionnaire grounded in pedagogical theory; factor analysis identified four reliable dimensions: perceived learning, engagement, believability, and challenge. Statistical analysis showed no significant difference between human and AEGIS scenarios across all dimensions, with AEGIS even scoring slightly higher on believability, likely due to the empirical validation of exploits that ensured realistic behavior.

In terms of efficiency, AEGIS reduced scenario development time from an estimated 30 person‑months (≈1,800 person‑days) to an average of 3.2 days (±0.5 days). Human effort was limited to defining high‑level goals and reviewing the final paths, while the automated pipeline handled discovery, validation, and path synthesis. The modular design—each stage implemented as an independent LLM agent plus supporting scripts—facilitates extension to new software, additional exploit sources, or larger networks.

The authors acknowledge limitations. Real‑time exploit execution is inherently slow, so scaling to thousands of hosts may become prohibitive without additional optimisations. Moreover, LLM‑driven web searches can miss newly published exploits or return false positives, which could affect coverage. Future work includes hybrid simulation (combining fast, abstracted models with selective real execution) and a feedback loop where validation outcomes refine the LLM’s search prompts, improving precision over time.

Overall, AEGIS demonstrates that integrating LLMs with white‑box validation and MCTS can automate the end‑to‑end creation of realistic, executable attack paths for large‑scale cyber‑defence training, dramatically cutting development effort while maintaining pedagogical quality. This represents a significant step forward in the automation of cyber‑exercise design and opens avenues for broader adoption in security education and assessment environments.