When Does Pairing Seeds Reduce Variance? Evidence from a Multi-Agent Economic Simulation

Machine learning systems appear stochastic but are deterministically random, as seeded pseudorandom number generators produce identical realisations across repeated executions. Standard evaluation practice typically treats runs across alternatives as independent and does not exploit shared sources of randomness. This paper analyses the statistical structure of comparative evaluation under shared random seeds. Under this design, competing systems are evaluated using identical seeds, inducing matched stochastic realisations and yielding strict variance reduction whenever outcomes are positively correlated at the seed level. We demonstrate these effects using an extended learning-based multi-agent economic simulator, where paired evaluation exposes systematic differences in aggregate and distributional outcomes that remain statistically inconclusive under independent evaluation at fixed budgets.

💡 Research Summary

This paper investigates a simple yet powerful experimental design for comparing algorithms or policy interventions in learning‑based multi‑agent simulations: using the same random seed for both alternatives, a technique the authors call “paired seed evaluation.” Because modern simulators rely on seeded pseudo‑random number generators to determine initial conditions, environment stochasticity, and learning noise, the same seed generates an identical stochastic trajectory for all runs. When the outcomes of two policies under a shared seed are positively correlated—a condition the authors empirically verify—the paired estimator of the average treatment effect has a variance that is strictly smaller than that of the conventional independent‑seed estimator.

The authors formalize the problem by defining Y(d, s) as the scalar outcome for policy d∈{0,1} under seed s. The estimand is Δ = E_s