Information Shapes Koopman Representation

The Koopman operator provides a powerful framework for modeling dynamical systems and has attracted growing interest from the machine learning community. However, its infinite-dimensional nature makes identifying suitable finite-dimensional subspaces challenging, especially for deep architectures. We argue that these difficulties come from suboptimal representation learning, where latent variables fail to balance expressivity and simplicity. This tension is closely related to the information bottleneck (IB) dilemma: constructing compressed representations that are both compact and predictive. Rethinking Koopman learning through this lens, we demonstrate that latent mutual information promotes simplicity, yet an overemphasis on simplicity may cause latent space to collapse onto a few dominant modes. In contrast, expressiveness is sustained by the von Neumann entropy, which prevents such collapse and encourages mode diversity. This insight leads us to propose an information-theoretic Lagrangian formulation that explicitly balances this tradeoff. Furthermore, we propose a new algorithm based on the Lagrangian formulation that encourages both simplicity and expressiveness, leading to a stable and interpretable Koopman representation. Beyond quantitative evaluations, we further visualize the learned manifolds under our representations, observing empirical results consistent with our theoretical predictions. Finally, we validate our approach across a diverse range of dynamical systems, demonstrating improved performance over existing Koopman learning methods. The implementation is publicly available at https://github.com/Wenxuan52/InformationKoopman.

💡 Research Summary

**
The paper tackles a fundamental challenge in modern Koopman learning: how to embed an infinite‑dimensional linear operator into a finite‑dimensional latent space that can be trained with deep neural networks. The authors argue that the difficulty stems from sub‑optimal representation learning—latent variables either become too compressed (losing predictive power) or too expressive (causing instability and mode collapse). To formalize this tension, they recast Koopman learning as a dynamic Information Bottleneck (IB) problem, where the latent variable zₙ₋₁ must be both compact with respect to the current state xₙ₋₁ and maximally predictive of the next state xₙ.

The theoretical contribution begins with two propositions. Proposition 1 shows a chain of mutual‑information inequalities: I(xₙ₋₁;xₙ) ≥ I(zₙ₋₁;xₙ) ≥ I(zₙ₋₁;zₙ). This reveals that the encoder inevitably discards some information about the future, and the linear Koopman transition further limits the amount of information that can be retained in the latent dynamics. Proposition 2 translates this information loss into an explicit error bound on the total‑variation distance between the true trajectory distribution and the one generated by the learned Koopman model. The bound is proportional to the sum over time of the gaps I(xₙ₋₁;xₙ) − I(zₙ₋₁;zₙ), establishing a direct quantitative link between information loss and multi‑step prediction error.

While mutual information I(zₜ;xₜ) quantifies how much of the original state is encoded, it does not capture the shape of the latent spectrum. Proposition 3 therefore decomposes I(zₜ;xₜ) into three spectral components: (i) a temporally coherent part associated with Koopman eigenvalues near the unit circle (|λ|≈1), (ii) a fast‑dissipating part linked to eigenvalues with |λ|<1, and (iii) a residual “noise” component that has no spectral support. This decomposition shows that a good Koopman representation must preserve information carried by the near‑unit‑modulus modes while allowing dissipative modes to decay naturally.

The authors identify two complementary information‑theoretic quantities that regulate the trade‑off. Mutual information encourages compression (simplicity) but, if over‑emphasized, drives the latent space to collapse onto a few dominant modes. Von Neumann entropy of the normalized covariance matrix ρ acts as a proxy for the effective dimensionality of the latent space; a higher entropy forces the information to spread across many eigenvectors, preventing mode collapse and promoting diversity.

Guided by these insights, they propose a Lagrangian objective: \