A General ReLearner: Empowering Spatiotemporal Prediction by Re-learning Input-label Residual

Prevailing spatiotemporal prediction models typically operate under a forward (unidirectional) learning paradigm, in which models extract spatiotemporal features from historical observation input and map them to target spatiotemporal space for future forecasting (label). However, these models frequently exhibit suboptimal performance when spatiotemporal discrepancies exist between inputs and labels, for instance, when nodes with similar time-series inputs manifest distinct future labels, or vice versa. To address this limitation, we propose explicitly incorporating label features during the training phase. Specifically, we introduce the Spatiotemporal Residual Theorem, which generalizes the conventional unidirectional spatiotemporal prediction paradigm into a bidirectional learning framework. Building upon this theoretical foundation, we design an universal module, termed ReLearner, which seamlessly augments Spatiotemporal Neural Networks (STNNs) with a bidirectional learning capability via an auxiliary inverse learning process. In this process, the model relearns the spatiotemporal feature residuals between input data and future data. The proposed ReLearner comprises two critical components: (1) a Residual Learning Module, designed to effectively disentangle spatiotemporal feature discrepancies between input and label representations; and (2) a Residual Smoothing Module, employed to smooth residual terms and facilitate stable convergence. Extensive experiments conducted on 11 real-world datasets across 14 backbone models demonstrate that ReLearner significantly enhances the predictive performance of existing STNNs.Our code is available on GitHub.

💡 Research Summary

The paper tackles a subtle yet pervasive problem in spatiotemporal forecasting: the mismatch between historical inputs and future labels, which the authors term “input‑label deviation.” Conventional Spatiotemporal Neural Networks (STNNs) follow a unidirectional learning paradigm—extracting spatiotemporal features from past observations and directly mapping them to future values. This approach assumes that the statistical dependencies present in the input persist unchanged in the target horizon. In practice, however, nodes with nearly identical histories can diverge into distinct future trajectories, dissimilar histories may converge to similar outcomes, and abrupt temporal shifts can break learned patterns. These phenomena degrade forecasting accuracy but have received limited attention in the literature.

To address this, the authors propose a bidirectional learning framework grounded in Gaussian Markov Random Field (GMRF) theory. By modeling the joint distribution of historical variables x and future variables y as a multivariate Gaussian with a sparse precision matrix, they derive a “Spatiotemporal Residual Theorem.” The theorem shows that the conditional expectation of y given x can be decomposed into a forward component (the usual prediction) plus a residual term that captures the systematic deviation between input and label representations. This residual is high‑dimensional and can be learned explicitly.

Building on the theorem, the paper introduces ReLearner, a lightweight plug‑in module that can be attached to any existing STNN. ReLearner consists of two sub‑modules:

1. Residual Learning Module – The input sequence is encoded into a latent representation Zₑ via an input encoder F_E, while the future labels (available only during training) are encoded into Zₕ using a label encoder F_R. The element‑wise difference Z_res = Zₑ – Zₕ constitutes the raw residual.

2. Residual Smoothing Module – To suppress noise and prevent unstable gradients, the raw residual passes through a propagation kernel applied L times with K distinct kernels. This operation smooths the residual across both spatial and temporal dimensions, yielding a refined residual ~Z_res.

The smoothed residual is then decoded by F_R, producing a correction term y_corr that is added to the base prediction y_base generated by the forward STNN, resulting in the final output ŷ = y_base + y_corr. Importantly, during inference only the forward path is needed; the label encoder is not required, preserving the original computational footprint.

The authors evaluate ReLearner on eleven real‑world datasets spanning traffic, weather, and energy domains, and integrate it with fourteen state‑of‑the‑art backbones (e.g., GCRN, MTGNN, ST‑Transformer). Across all settings, ReLearner yields consistent improvements: average reductions of 10% in MAE/RMSE and up to 21.18% gain on the most challenging dataset. Parameter overhead is modest (≈5% of total model size), and training remains stable thanks to the smoothing stage.

Strengths

• Theoretical novelty: The GMRF‑based residual theorem provides a principled justification for modeling input‑label deviation, a concept rarely formalized before.
• Universal applicability: ReLearner’s design is architecture‑agnostic; it can be attached to convolutional, recurrent, or transformer‑based STNNs without redesigning the core model.
• Empirical robustness: Extensive benchmarks demonstrate that the method works across diverse domains and backbone complexities.
• Efficiency: The added modules introduce minimal extra parameters and computational cost, making the approach practical for many real‑time scenarios.

Weaknesses / Open Issues

• Dependence on high‑quality label embeddings: The backward path relies on accurate label representations; noisy or sparse labels could propagate errors into the residual learning stage.
• Hyper‑parameter sensitivity: The number of kernels (K) and smoothing layers (L) affect both performance and runtime; the paper provides limited guidance on selecting these values for resource‑constrained deployments.
• OOD generalization: While the authors claim relevance to both IID and OOD settings, explicit experiments on severe distribution shifts (e.g., sensor failures, policy changes) are absent, leaving the robustness of ReLearner under extreme shifts uncertain.
• Interpretability of residuals: Although the residual term is mathematically defined, the paper offers few visualizations or analyses linking specific residual patterns to real‑world phenomena (e.g., traffic incidents).

Potential Impact and Future Directions
ReLearner opens a new line of research that treats future labels as an auxiliary source of information during training, rather than a mere supervision signal. This perspective could inspire extensions such as: (i) adversarial training where synthetic label embeddings are generated to augment scarce label data; (ii) adaptive smoothing kernels learned jointly with the backbone to automatically balance bias‑variance trade‑offs; (iii) multi‑modal label integration (e.g., satellite imagery, textual reports) to enrich the residual signal; and (iv) robustness mechanisms that detect and down‑weight noisy label embeddings. Moreover, the residual framework may be combined with existing OOD detection techniques to create models that both anticipate and correct for distributional shifts.

In summary, the paper presents a theoretically grounded, practically effective, and broadly applicable solution to a previously under‑explored limitation of spatiotemporal forecasting models. By explicitly learning and smoothing the input‑label residual, ReLearner consistently improves prediction accuracy across a wide range of tasks while maintaining a lightweight footprint, marking a significant step forward for the field.