Variational Markov chain mixtures with automatic component selection

Markov state modeling has gained popularity in various scientific fields since it reduces complex time-series data sets into transitions between a few states. Yet common Markov state modeling frameworks assume a single Markov chain describes the data, so they suffer from an inability to discern heterogeneities. As an alternative, this paper models time-series data using a mixture of Markov chains, and it automatically determines the number of mixture components using the variational expectation-maximization algorithm.Variational EM simultaneously identifies the number of Markov chains and the dynamics of each chain without expensive model comparisons or posterior sampling. As a theoretical contribution, this paper identifies the natural limits of Markov state mixture modeling by proving a lower bound on the classification error. It then presents numerical experiments where variational EM achieves performance consistent with the theoretically optimal error scaling. The experiments are based on synthetic and observational data sets including Last.fm music listening, ultramarathon running, and gene expression. In each of the three data sets, variational EM leads to the identification of meaningful heterogeneities.

💡 Research Summary

The paper addresses a fundamental limitation of traditional Markov state modeling (MSM), which assumes that a single Markov chain generates all observed trajectories. Modern datasets—large collections of short, noisy, and heterogeneous time‑series—often violate this “one‑chain” assumption, leading to poor representation of underlying dynamics. To overcome this, the authors propose a finite‑state Markov chain mixture (MCM) model in which each trajectory is generated by one of K latent Markov chains. The mixture is defined by a prior over chain labels μ, chain‑specific initial state distributions ν_i, and transition matrices P_i.

A central contribution is the development of a variational expectation‑maximization (VEM) algorithm tailored to this setting. Unlike the classical EM algorithm, VEM introduces a variational distribution q(Z) over the latent labels and maximizes the evidence lower bound (ELBO). In the E‑step, q_i is updated proportionally to μ_i exp( E_{Y|Z=i}