Learning in Structured Stackelberg Games

We initiate the study of structured Stackelberg games, a novel form of strategic interaction between a leader and a follower where contextual information can be predictive of the follower’s (unknown) type. Motivated by applications such as security games and AI safety, we show how this additional structure can help the leader learn a utility-maximizing policy in both the online and distributional settings. In the online setting, we first prove that standard learning-theoretic measures of complexity do not characterize the difficulty of the leader’s learning task. Notably, we find that there exists a learning-theoretic measure of complexity, analogous to the Littlestone dimension in online classification, that tightly characterizes the leader’s instance-optimal regret. We term this the Stackelberg-Littlestone dimension, and leverage it to provide a provably optimal online learning algorithm. In the distributional setting, we provide analogous results by showing that two new dimensions control the sample complexity upper- and lower-bound.

💡 Research Summary

The paper introduces “structured Stackelberg games,” a generalization of the classic Stackelberg framework in which both the leader and the follower observe a contextual signal z before the leader commits to a mixed strategy. The follower belongs to one of K possible types, each type being characterized by its own utility function u_f(i). The mapping from contexts to follower types is unknown but assumed to belong to a hypothesis class H known to the leader. The leader’s goal is to maximize her own utility u, which depends on the context, her mixed action, and the follower’s best response.

The authors study two learning regimes. In the online regime, contexts and follower types may be chosen adversarially. Prior work showed that if either contexts or types are i.i.d., no‑regret learning is possible, but when both are adversarial the regret can be linear. The paper shows that this impossibility can be avoided if the leader knows that the context‑type mapping lies in H. However, standard complexity measures from online multiclass classification, such as the Littlestone dimension, are insufficient because they ignore the leader’s utility structure. To address this, the authors define a new combinatorial parameter, the Stackelberg‑Littlestone (SL) dimension. The SL dimension is based on shattered trees that incorporate both the context‑type mapping and the leader’s payoff differences. They prove that for any hypothesis class, the optimal regret scales as Θ(√{d T}), where d is the SL dimension (Theorem 3.10 and 3.11). Moreover, they present an explicit online algorithm (Algorithm 1) that attains this bound, thereby achieving instance‑optimal regret.

In the distributional (PAC) setting, contexts are drawn i.i.d. from an unknown distribution, while follower types are generated by a fixed hypothesis h∈H. The learner receives labeled examples (z, h(z)) and must output a policy that maximizes expected utility. The paper introduces two new dimensions: the γ‑SN (mistake‑sensitivity) dimension, which yields a lower bound on the number of samples needed, and the γ‑SG (growth) dimension, which provides an upper bound on sample complexity. Theorems 4.4 and 4.7 show that the sample complexity is Θ((γ‑SG)/ε² · log (1/δ)) up to constant factors, and an improper learning algorithm (Algorithm 2) achieves this rate with high probability.

The work also connects to several related areas. It subsumes prior results on learning in Stackelberg games with a single follower type, extends online results for multiple types, and improves upon earlier impossibility results by leveraging structural assumptions on the context‑type mapping. The authors discuss applications to wildlife protection, security patrols, AI red‑team testing, and budget‑aware fine‑tuning of large language models, where side information is predictive of the adversary’s type. Computational considerations are addressed: the SL‑based algorithm requires solving a convex optimization over the leader’s simplex, which is tractable for moderate action spaces, and the PAC algorithm can be implemented via empirical risk minimization with a suitable surrogate loss.

Finally, the paper shows that the theoretical framework extends to related problems such as Bayesian persuasion with public states and auction bidding with side information, and it provides a clean separation between the difficulty of learning the context‑type mapping (captured by the new dimensions) and the difficulty of optimizing against the learned mapping (captured by the leader’s payoff structure). Overall, the paper delivers a comprehensive theory that precisely characterizes both regret in the online adversarial setting and sample complexity in the stochastic setting for structured Stackelberg games, introducing novel complexity measures that are likely to influence future research at the intersection of game theory, online learning, and AI safety.