Empirical parameterization of the Elo Rating System

This study aims to provide a data-driven approach for empirically tuning and validating rating systems, focusing on the Elo system. Well-known rating frameworks, such as Elo, Glicko, TrueSkill systems, rely on parameters that are usually chosen based on probabilistic assumptions or conventions, and do not utilize game-specific data. To address this issue, we propose a methodology that learns optimal parameter values by maximizing the predictive accuracy of match outcomes. The proposed parameter-tuning framework is a generalizable method that can be extended to any rating system, even for multiplayer setups, through suitable modification of the parameter space. Implementation of the rating system on real and simulated gameplay data demonstrates the suitability of the data-driven rating system in modeling player performance.

💡 Research Summary

The paper presents a data‑driven methodology for empirically tuning the parameters of the Elo rating system, with the aim of improving predictive accuracy of match outcomes. Traditional Elo implementations rely on conventionally chosen constants—most notably the K‑factor, which governs how much a player’s rating changes after a game. While many platforms adopt a piecewise‑decreasing K‑factor based on the number of games played, the specific values (Kₐ, K_b, K_c) and the thresholds (n_c₁, n_c₂) that separate early, middle, and late stages are typically set arbitrarily or based on historical practice rather than on actual game data.

The authors propose to treat these parameters as hyper‑parameters that can be optimized by maximizing the out‑of‑sample predictive performance of a classification model that predicts the winner from pre‑match rating differences. The workflow is as follows: (1) define a grid of candidate K‑factor triples and threshold pairs; (2) for each candidate, compute player ratings over the entire dataset using the standard Elo update rule; (3) train a logistic regression model (or any other classifier) on the rating differences to predict match outcomes; (4) evaluate the model using the F1‑score; (5) select the parameter set that yields the highest F1‑score. Because the only predictor is the rating difference, any improvement in the F1‑score directly reflects a better calibration of the underlying rating dynamics.

Two datasets are used for empirical validation. The first is a synthetic dataset generated from 7 bots playing a two‑player, three‑dice Ludo variant, comprising 184,000 games. The second is a real‑world dataset collected from the “Games24x7” platform, containing 4,640,765 matches among 320,978 distinct players over a 2.5‑month period. Four K‑factor configurations are examined: (60,30,16) – the baseline used by many online chess services; (30,30,30) – a constant K for control; (30,16,8) – a scaled‑down version favoring stability; and (100,50,25) – a scaled‑up version for higher responsiveness. Three threshold schemes are tested: fixed (5,10) games, and two percentile‑based schemes derived from the distribution of games per player (