Sparse Equation Matching: A Derivative-Free Learning for General-Order Dynamical Systems

Equation discovery is a fundamental learning task for uncovering the underlying dynamics of complex systems, with wide-ranging applications in areas such as brain connectivity analysis, climate modeling, gene regulation, and physical simulation. However, many existing approaches rely on accurate derivative estimation and are limited to first-order dynamical systems, restricting their applicability in real-world scenarios. In this work, we propose Sparse Equation Matching (SEM), a unified framework that encompasses several existing equation discovery methods under a common formulation. SEM introduces an integral-based sparse regression approach using Green’s functions, enabling derivative-free estimation of differential operators and their associated driving functions in general-order dynamical systems. The effectiveness of SEM is demonstrated through extensive simulations, benchmarking its performance against derivative-based approaches. We then apply SEM to electroencephalographic (EEG) data recorded during multiple oculomotor tasks, collected from 52 participants in a brain-computer interface experiment. Our method identifies active brain regions across participants and reveals task-specific connectivity patterns. These findings offer valuable insights into brain connectivity and the underlying neural mechanisms.

💡 Research Summary

This paper tackles the problem of discovering governing differential equations from noisy time‑series data, a task that becomes especially challenging for higher‑order dynamical systems where accurate derivative estimation is unstable. The authors introduce Sparse Equation Matching (SEM), a unified framework that eliminates the need for explicit derivative computation by leveraging Green’s‑function based integral formulations and sparse regression.

The core idea is to rewrite a general K‑th order ordinary differential equation (ODE)
(P_{K}X_i(t)=f_i\bigl(X(t),t\bigr)), where (P_{K}=d^{K}/dt^{K}+\sum_{l=1}^{K-1}\omega_{il}d^{l}/dt^{l}), into an equivalent integral equation using the appropriate Green’s function. This transformation embeds both the linear differential operator and the unknown driving function into a single operator (F_{f,\omega}). By integrating the squared norm of (F_{f,\omega}) over the observation window, the authors obtain a loss functional that is zero only at the true parameters, even when the underlying trajectories and their derivatives are not directly observed.

Implementation proceeds in three stages. First, the noisy observations (Y_{ij}=X_i(t_j)+\varepsilon_{ij}) are smoothed via reproducing‑kernel Hilbert space (RKHS) regression, typically in a Sobolev space whose kernel order matches the ODE order K. This yields smooth estimates (\hat X_i(t)) and, crucially, analytically tractable derivatives up to order K. Second, the integral loss is minimized with respect to the operator coefficients (\omega) and the functional form of (f). The authors impose an (L_1) sparsity penalty (LASSO/Elastic Net) on the coefficients of a pre‑specified library of candidate functions, encouraging parsimonious models that reflect the expectation of limited interactions in real systems. Third, hyper‑parameters (regularization strength, kernel bandwidth) are selected by generalized cross‑validation and model‑selection criteria such as AIC/BIC.

Theoretical contributions include: (i) proof that Sobolev‑kernel smoothing guarantees the required differentiability of (\hat X); (ii) demonstration that the integral loss retains identifiability of ((\omega,f)) without explicit derivative data; and (iii) analysis of consistency and robustness under additive white noise.

Empirical evaluation comprises two parts. In synthetic experiments, the authors generate first‑, second‑, and third‑order ODE systems with known sparse structures and add Gaussian noise at various signal‑to‑noise ratios. Compared against state‑of‑the‑art derivative‑based methods such as SINDy and PDE‑Fit, SEM consistently achieves lower mean‑squared error, especially for second‑ and third‑order systems where derivative estimation fails.

The second part applies SEM to real electroencephalographic (EEG) recordings from 52 participants performing three oculomotor tasks (eye blinking, horizontal, and vertical movements). Each EEG channel is treated as a state variable, and a second‑order model is fitted. SEM successfully predicts unseen EEG segments and, more importantly, extracts interpretable connectivity patterns: (a) task‑specific active regions in frontal and parietal cortices, (b) a common core network shared across tasks, and (c) task‑specific higher‑order feedback loops that are invisible to traditional Granger causality or Dynamic Causal Modeling analyses. These findings corroborate known neurophysiological literature while revealing novel dynamical motifs.

Advantages of SEM are threefold: (1) it is derivative‑free, making it robust to measurement noise; (2) it naturally accommodates general‑order dynamics, extending equation discovery beyond the usual first‑order setting; and (3) sparsity enforcement yields models that are both accurate and scientifically interpretable. Limitations include the need for a suitable Green’s function (which may require domain expertise) and computational scaling issues because kernel matrices grow quadratically with the number of time points. The authors suggest future work on random‑feature approximations, Nyström methods, and extensions to nonlinear differential operators or time‑varying parameters.

In summary, Sparse Equation Matching provides a powerful, unified, and derivative‑free approach for learning governing equations of arbitrary order from noisy data, with demonstrated superiority over existing methods in both synthetic benchmarks and a challenging real‑world EEG connectivity study.