Causation entropy from symbolic representations of dynamical systems

Identification of causal structures and quantification of direct information flows in complex systems is a challenging yet important task, with practical applications in many fields. Data generated by dynamical processes or large-scale systems are often symbolized, either because of the finite resolution of the measurement apparatus, or because of the need of statistical estimation. By algorithmic application of causation entropy, we investigated the effects of symbolization on important concepts such as Markov order and causal structure of the tent map. We uncovered that these quantities depend nonmontonically and, most of all, sensitively on the choice of symbolization. Indeed, we show that Markov order and causal structure do not necessarily converge to their original analog counterparts as the resolution of the partitioning becomes finer.

💡 Research Summary

The paper tackles the fundamental problem of inferring causal relationships and quantifying direct information flows in complex dynamical systems, with particular attention to the effects of symbolization—i.e., the discretization of continuous state spaces into a finite alphabet. Symbolization is ubiquitous in practice because measurement devices have finite resolution and because statistical estimation often requires discrete data. The authors ask a subtle but crucial question: how does the choice of partition (the way the phase space is divided) influence the inferred Markov order and the causal structure of the underlying process?

To answer this, they introduce an information‑theoretic measure called causation entropy (CSE), defined as a conditional mutual information
(C_{J\to t}=I(S_J;S_t\mid S_{t\setminus J})),
where (S_J) denotes a set of past symbols indexed by (J) and (S_t) the current symbol. CSE is non‑negative and strictly positive exactly when the selected past symbols provide additional information about the present beyond what is already supplied by the remaining past. This property makes CSE a natural tool for both (i) determining the Markov order—the smallest (k) such that (C_{t-k\to t}=0)—and (ii) identifying the causal parents (P_t) of the current state, i.e., the minimal subset of past indices for which (C_{t\setminus P_t\to t}=0) while no proper subset satisfies the same condition.

The authors develop a two‑stage algorithm to estimate (P_t). First, they perform an aggregative discovery step, selecting the past time index that maximizes the ordinary mutual information with the present. Then, they iteratively add indices that maximize the conditional mutual information given the already selected set, stopping when the added contribution falls to zero. Under the standard faithfulness (stability) assumption, this procedure yields the minimal set that maximizes CSE, thereby providing a principled and computationally tractable way to recover the causal structure without exhaustive testing of all subsets.

To illustrate the theory, the paper focuses on the tent map, a piecewise linear chaotic map defined by (x_{t+1}=1-2|x_t-0.5|). The map is analytically simple, yet it exhibits nontrivial symbolic dynamics when the unit interval is partitioned into a finite number of subintervals. The authors systematically vary the location of the partition boundaries, keeping the number of symbols fixed, and compute the exact stationary joint probabilities of the resulting symbolic process using the invariant measure of the tent map. With these probabilities they evaluate CSE for various candidate past sets, thereby determining the Markov order and causal parents for each partition.

The results are striking: both the Markov order and the causal structure depend on the partition in a highly non‑monotonic and sensitive manner. Small shifts of a boundary can cause the inferred Markov order to jump from 1 (the true order of the original continuous map) to 3 or 4, and the set of causal parents can change from a single lag (t‑1) to a combination of several lags (e.g., t‑4, t‑3, t‑1). Moreover, refining the partition (making the bins narrower) does not guarantee convergence to the original analog values. Even in the limit of arbitrarily fine partitions, certain boundary placements continue to produce higher‑order symbolic processes, demonstrating that the symbolic representation can retain “memory” that is absent in the underlying continuous dynamics.

These findings have several important implications. First, they caution against the naïve use of discretization in causal inference: the act of symbolizing data can create spurious causal links or mask genuine ones, depending on how the phase space is divided. Second, the work validates causation entropy as a robust, low‑dimensional statistic that aggregates high‑dimensional joint probabilities, thereby mitigating the curse of dimensionality that plagues traditional conditional independence tests. Third, the two‑stage algorithm provides a practical workflow for researchers dealing with large‑scale time series where exhaustive conditional independence testing is infeasible.

The paper also discusses limitations. The analysis is confined to a one‑dimensional map with a known invariant measure, which allows exact probability calculations; extending the approach to higher‑dimensional, noisy, or non‑stationary systems will require estimation techniques and may introduce additional challenges. Nevertheless, the core message—that symbolization choices fundamentally alter inferred causal structures—is likely to hold broadly. Future work could explore adaptive partitioning strategies that preserve causal information, or integrate CSE with machine‑learning frameworks for automated causal discovery in real‑world datasets.

In summary, the study provides a rigorous, information‑theoretic framework for assessing how discretization impacts causal inference, demonstrates the nontrivial dependence of Markov order and causal parents on partition design using the tent map, and offers practical algorithms that could be valuable for scientists across disciplines who rely on symbolic representations of dynamical data.