Differentially Private Data-Driven Markov Chain Modeling

Markov chains model a wide range of user behaviors. However, generating accurate Markov chain models requires substantial user data, and sharing these models without privacy protections may reveal sensitive information about the underlying user data. We introduce a method for protecting user data used to formulate a Markov chain model. First, we develop a method for privatizing database queries whose outputs are elements of the unit simplex, and we prove that this method is differentially private. We quantify its accuracy by bounding the expected KL divergence between private and non-private queries. We extend this method to privatize stochastic matrices whose rows are each a simplex-valued query of a database, which includes data-driven Markov chain models. To assess their accuracy, we analytically bound the change in the stationary distribution and the change in the convergence rate between a non-private Markov chain model and its private form. Simulations show that under a typical privacy implementation, our method yields less than 2% error in the stationary distribution, indicating that our approach to private modeling faithfully captures the behavior of the systems we study.

💡 Research Summary

This paper tackles the privacy challenges inherent in data‑driven Markov chain modeling by introducing a differential privacy (DP) framework that protects the underlying user data while preserving the utility of the resulting transition matrix. The authors observe that transition probabilities in a Markov chain are typically estimated from large databases of user actions; releasing the raw matrix can inadvertently reveal sensitive individual behavior. To address this, they develop a two‑stage mechanism based on the Dirichlet distribution, which is naturally suited for outputs that must lie on the unit simplex (i.e., probability vectors).

In the first stage, the paper extends the Dirichlet mechanism originally proposed by Gohari et al. (2021) to handle queries that are functions of a database. Given a query output (p \in \Delta^n) (the unit simplex), the mechanism samples a noisy vector (\tilde p) from a Dirichlet distribution with concentration parameter (k p). The scalar (k>0) controls the privacy‑accuracy trade‑off: larger (k) yields less noise and higher accuracy, while smaller (k) provides stronger privacy. By partitioning the output space into a “good’’ region (\Omega_1) (all components above a threshold (\gamma)) and a “bad’’ region (\Omega_2), the authors prove that the mechanism satisfies ((\varepsilon,\delta))-DP in the conventional sense, where (\varepsilon) is derived from the concentration parameter and (\delta) bounds the probability of landing in (\Omega_2). This construction guarantees that every noisy output remains a valid probability vector, avoiding the projection step required by Gaussian or Laplace mechanisms that would otherwise break the simplex constraints.

The second stage lifts the vector‑level mechanism to full stochastic matrices. Since each row of a transition matrix is an independent probability vector, the authors apply the Dirichlet mechanism row‑wise, leveraging the composition property of DP to obtain overall privacy guarantees for the entire matrix. Theorem 3 formalizes this composition and provides explicit formulas linking the per‑row privacy budget to the global ((\varepsilon,\delta)) parameters.

Accuracy analysis proceeds on two fronts. First, Theorem 2 and Corollary 1 bound the expected Kullback‑Leibler (KL) divergence between a true query vector and its privatized counterpart, showing it scales as (O(1/k)). This quantifies how much information is lost at the vector level. Second, the authors study the impact on the Markov chain itself. Theorem 4 bounds the (L_1) distance between the stationary distribution (\pi) of the original chain and (\tilde\pi) of the privatized chain, using the matrix norm and the KL bound from the first stage. Theorem 5 bounds the change in the ergodicity coefficient (a proxy for the convergence rate) by relating it to the spectral gap of the original matrix and the magnitude of the added Dirichlet noise. Together, these results give a clear, quantitative picture of how privacy parameters affect both asymptotic behavior (steady‑state distribution) and transient dynamics (mixing speed).

Empirical validation uses two real‑world datasets: (1) a university class grade distribution and (2) New York City taxi trip data. For each, the authors compute the empirical transition matrix, apply the Dirichlet mechanism with parameters yielding ((\varepsilon,\delta) = (3.73, 6\times10^{-6})), and then compare the privatized stationary distribution and convergence rate to the originals. The average relative error in the stationary distribution is under 2 %, and the change in the ergodicity coefficient is less than 0.03, indicating negligible impact on mixing speed. These results demonstrate that strong privacy can be achieved without sacrificing practical modeling accuracy.

Key contributions of the work are:

1. A unified Dirichlet‑based DP mechanism that handles both simplex‑valued queries and full stochastic matrices, eliminating the need for post‑processing projections.
2. Rigorous theoretical bounds on KL divergence, stationary‑distribution deviation, and convergence‑rate alteration, providing practitioners with explicit privacy‑accuracy trade‑off formulas.
3. Validation on large‑scale, real‑world data showing that the method delivers sub‑2 % error in steady‑state behavior under a stringent privacy budget.

Limitations include the assumption of row‑wise independence; if the transition matrix has structural constraints (e.g., symmetry, detailed balance) that couple rows, independent Dirichlet noise may violate those constraints. Moreover, selecting the concentration parameter (k) to balance privacy and utility remains a practical challenge, especially for high‑dimensional state spaces where the required (k) may become large. Future work could explore joint noise mechanisms that respect inter‑row constraints, adaptive tuning of (k) based on data characteristics, and extensions to streaming or online settings where the transition matrix evolves over time.