What is the AGI in Offensive Security ?

What is the AGI in Offensive Security? One can break it down into two questions : (1) any offensive security tasks could be reduced into symbolic language manipulation (language representation + reasoning), (2) powerful language model (LLM) are enough to “deal with” any symbolic language manipulation. This paper can formally model a target system as a state machine and a hacker as an interactive symbolic agent. And it shows that every interaction in an offensive engagement can be encoded as a finite string. This paper provides definitions, short lemmas, and open discussion.

💡 Research Summary

The paper asks a seemingly simple question—what is the nature of offensive security?—and answers it by reducing every offensive security task to symbolic language manipulation and then arguing that large language models (LLMs) are, in principle, capable of handling such manipulation. The authors construct a formal framework that models a target digital system as a state machine and the attacker as an interactive symbolic agent. They then demonstrate that any finite interaction between the two can be encoded as a finite string, and that the attacker’s policy itself can be represented as a finite description. From these observations they derive a series of lemmas that collectively claim: (1) the entire transcript of an attack can be flattened into a single symbolic sequence; (2) any countable set of symbols can be tokenized into a finite vocabulary suitable for sequence models; (3) a computable attacker policy can be encoded as a finite string; and (4) a sufficiently large LLM can learn to approximate the conditional distribution of the attacker’s next move given the history of outputs. The paper concludes that, if the modeling assumptions hold, an LLM could serve as an “AGI of hacking” by either imitating the attacker’s behavior or synthesizing the policy code itself.

The target system model M = (S, Σ, O, g) follows standard formal methods: S is the (potentially infinite but countable) set of system states, Σ the finite input alphabet (e.g., bytes), O the finite output alphabet, and g a (partial) transition function that maps a state and an input string to a new state and an output string. This abstraction captures any digital computer, as any algorithmic process can be simulated by a Turing machine, which is itself a state machine.

The attacker is defined as a policy π : (O* )* → Σ* that, given the entire history of system outputs, produces the next input. π may be deterministic or probabilistic, may maintain internal state, and can be implemented as any computable algorithm. The interaction proceeds in a loop: the attacker sends an input, the system transitions and emits an output, the attacker observes the output and decides the next input, and so on, until termination.

Lemma 1.1 proves that any finite interaction transcript T = (i₁, o₁, …, iₙ, oₙ) can be encoded as a string over a new alphabet Γ by inserting delimiters (e.g., “#” and “$”) between encoded inputs and outputs. Because Σ and O are finite, such an encoding is always possible and injective. Lemma 1.2 generalizes this by showing that any countable symbol set X can be mapped injectively into strings over a finite token set V, using standard numeral encodings. These two lemmas establish that the entire hacking episode can be treated as a single sequence of symbols, making it amenable to token‑based models.

Lemma 2.1 states that any computable policy π admits a finite description P (e.g., source code, binary, or Turing‑machine description) over a fixed alphabet Υ. This follows from basic computability theory: every algorithm can be represented by a finite string.

Lemma 2.2 connects the formalism to LLMs. Because π defines a deterministic conditional distribution δπ(h) that maps a history h to a single next input, a sufficiently large language model with parameters θ can be trained so that its conditional probability Prθ(x | h) approximates δπ(h)(x). In the limit of unlimited data, model capacity, and training time, the LLM can reproduce the attacker’s decision rule. The authors also note that, since π itself can be encoded as a string, an LLM could in principle generate the policy code rather than merely imitate its actions.

The paper’s contributions are primarily theoretical. By casting offensive security entirely in terms of symbolic strings, the authors provide a clean bridge to the existing literature on formal verification, model checking, and sequence modeling. This abstraction suggests new avenues for automated analysis: attack logs could be normalized into strings, tokenized, and fed into LLMs for anomaly detection, attack synthesis, or defensive policy generation.

However, several practical limitations are evident. Real‑world systems exhibit non‑determinism, timing side‑channels, and continuous‑valued phenomena that are not naturally captured by a purely discrete state‑transition model. Human attackers bring intuition, creativity, and strategic reasoning that may not be fully reducible to a computable function, especially when dealing with incomplete information or adaptive defenses. Moreover, the lemmas guarantee existence but do not provide constructive encoding schemes, tokenization strategies, or data‑efficiency analyses required for actual implementation. Training an LLM to faithfully replicate sophisticated attack policies would demand massive, high‑quality datasets of attack transcripts, which are scarce and often sensitive.

In summary, the paper offers a compelling formal perspective: offensive security can be expressed as symbolic language manipulation, and large language models possess the theoretical capacity to learn and reproduce such manipulation. While the framework is elegant and opens promising research directions—such as LLM‑driven attack simulation, automated exploit generation, and formal reasoning about security policies—realizing these ideas will require addressing the gap between idealized discrete models and the messy, probabilistic nature of real cyber‑physical environments, as well as developing practical pipelines for data collection, encoding, and model training.