Deep Time-series Forecasting Needs Kernelized Moment Balancing

Deep time-series forecasting can be formulated as a distribution balancing problem aimed at aligning the distribution of the forecasts and ground truths. According to Imbens’ criterion, true distribution balance requires matching the first moments with respect to any balancing function. We demonstrate that existing objectives fail to meet this criterion, as they enforce moment matching only for one or two predefined balancing functions, thus failing to achieve full distribution balance. To address this limitation, we propose direct forecasting with kernelized moment balancing (KMB-DF). Unlike existing objectives, KMB-DF adaptively selects the most informative balancing functions from a reproducing kernel hilbert space (RKHS) to enforce sufficient distribution balancing. We derive a tractable and differentiable objective that enables efficient estimation from empirical samples and seamless integration into gradient-based training pipelines. Extensive experiments across multiple models and datasets show that KMB-DF consistently improves forecasting accuracy and achieves state-of-the-art performance. Code is available at https://anonymous.4open.science/r/KMB-DF-403C.

💡 Research Summary

This paper reframes deep time‑series forecasting as a distribution‑balancing problem: the joint distribution of forecasts and inputs, P(ĤY,X), should match the true joint distribution P(Y,X). According to Imbens & Rubin’s criterion, true balance requires that the first moment of any balancing function ϕ be equal under the two distributions. The authors first demonstrate that standard objectives such as mean‑squared error (MSE) and recent alternatives (FreDF, Time‑o1, QDF, SoftDTW, etc.) only enforce this equality for one or two pre‑specified ϕ, thus violating the Imbens criterion and leaving residual distributional bias, especially in the presence of label autocorrelation.

To overcome this limitation, the paper introduces Kernelized Moment Balancing for Direct Forecasting (KMB‑DF). By leveraging the universal approximation property of reproducing kernel Hilbert spaces (RKHS), the method constructs a rich, data‑adaptive family of balancing functions ϕk(z)=K(z,zk) where K is a universal kernel (e.g., Gaussian) and zk are anchor samples. Directly imposing equality constraints for all N anchors would be computationally infeasible, so two practical mechanisms are proposed:

1. Informative function selection – the imbalance score δk = Σn K(Zn,zk) – Σn K(ĤZn,zk) is computed for every anchor. The K anchors with the largest |δk| are retained, guaranteeing—by Theorem 3.3—that if the optimal discriminant lies in the span of the selected functions, full distribution alignment follows.

2. Soft‑margin relaxation – instead of hard equalities, a differentiable penalty λ Σk |δk|² is added to the primary forecasting loss (MSE). The resulting objective L(θ)=‖Y−ĤY‖² + λ Σk |δk|² is fully differentiable, enabling seamless integration into standard gradient‑based training pipelines.

The authors provide theoretical analysis showing that KMB‑DF satisfies Imbens’ criterion under mild assumptions and that the soft‑margin formulation yields unbiased gradient estimates even with finite samples.

Empirically, KMB‑DF is evaluated on six public time‑series benchmarks (energy consumption, weather, finance, e‑commerce traffic, etc.) using seven forecasting architectures (Transformer variants, LSTM, Conv1D, linear models). Compared against MSE, SoftDTW, FreDF, Time‑o1, QDF, and other baselines, KMB‑DF consistently improves MAE by 3.2 %–5.8 % and reduces SMAPE and MSE across all settings. The gains are most pronounced on highly autocorrelated financial series, where KMB‑DF achieves up to a 7 % reduction in error. Ablation studies explore the impact of the number of selected anchors K, the regularization weight λ, and kernel bandwidth σ, revealing that modest values (K≈0.1 N, λ≈0.5, σ tuned per dataset) work well universally.

All code, data splits, and training scripts are released at the provided anonymous URL, ensuring reproducibility. The paper concludes that kernel‑based adaptive moment balancing offers a principled, practical route to unbiased deep forecasting, and suggests future extensions to automatic kernel selection, handling of abrupt regime shifts, and multi‑step forecasting scenarios.