Online time series prediction using feature adjustment

Time series forecasting is of significant importance across various domains. However, it faces significant challenges due to distribution shift. This issue becomes particularly pronounced in online deployment scenarios where data arrives sequentially, requiring models to adapt continually to evolving patterns. Current time series online learning methods focus on two main aspects: selecting suitable parameters to update (e.g., final layer weights or adapter modules) and devising suitable update strategies (e.g., using recent batches, replay buffers, or averaged gradients). We challenge the conventional parameter selection approach, proposing that distribution shifts stem from changes in underlying latent factors influencing the data. Consequently, updating the feature representations of these latent factors may be more effective. To address the critical problem of delayed feedback in multi-step forecasting (where true values arrive much later than predictions), we introduce ADAPT-Z (Automatic Delta Adjustment via Persistent Tracking in Z-space). ADAPT-Z utilizes an adapter module that leverages current feature representations combined with historical gradient information to enable robust parameter updates despite the delay. Extensive experiments demonstrate that our method consistently outperforms standard base models without adaptation and surpasses state-of-the-art online learning approaches across multiple datasets.

💡 Research Summary

The paper tackles two pervasive challenges in online time‑series forecasting: (1) distribution shift caused by evolving latent factors, and (2) delayed feedback inherent to multi‑step predictions where true targets become available only after several steps. Existing online learning methods typically focus on selecting a subset of model parameters (final‑layer weights, adapters) and on devising update strategies (recent mini‑batches, replay buffers, EMA). The authors argue that these approaches miss the root cause of drift – changes in the underlying latent variables that generate the series – and propose to adapt directly in the feature space that encodes those latent factors.

The proposed method, ADAPT‑Z (Automatic Delta Adjustment via Persistent Tracking in Z‑space), decomposes any forecasting model into an encoder f (producing latent features zₜ) and a prediction head g. Instead of updating g or the encoder weights, ADAPT‑Z learns a correction term δₜ such that g(zₜ + δₜ) ≈ yₜ. The correction is generated by a lightweight MLP adapter that takes two inputs: the current feature vector zₜ and a historical gradient vector that aggregates past feature‑gradient information. Because the adapter predicts δₜ from zₜ available at the current time step, it sidesteps the k‑step delay problem: the model can produce a corrected forecast immediately, while the true target will arrive later.

To handle the high variance of single‑sample gradients, the authors compute a batch‑averaged loss over a window that ends k steps before the current time, and use the resulting averaged gradient as the historical gradient input. The adapter’s architecture uses a dual‑path design: separate linear projections for zₜ and the gradient, followed by summation and two additional linear layers, which mitigates scale mismatches between features and gradients. During deployment, a k‑step delayed online gradient descent is performed: when the true value at time t arrives, the loss associated with the prediction made at t − k is back‑propagated to update both the adapter and, optionally, the final linear layer of the base model.

Experiments cover 13 publicly available time‑series datasets (four ETT, four PEMS, and five others such as weather, solar, traffic, electricity, exchange) and three state‑of‑the‑art forecasting backbones: iTransformer, SOFTS, and TimesNet. The authors adopt a realistic chronological split (60 % train, 10 % validation, 30 % test) and evaluate 12‑, 24‑, and 48‑step horizons. Baselines include recent online adaptation methods DSOF, Proceed, ADCSD, and SOLID, each applied to the same backbones. Across all settings, ADAPT‑Z consistently outperforms the baselines, achieving average MSE reductions of 5–12 % and showing particular robustness during periods of abrupt distribution change.

A notable side finding is the “learn‑to‑adapt” phenomenon: pre‑training the adapter with feature‑gradient information enables the model to partially adapt to drift without any online parameter updates, suggesting that the adapter can internalize a generic correction policy.

In summary, ADAPT‑Z introduces a conceptually simple yet powerful paradigm: adapt the latent feature space rather than the model parameters. By leveraging current features and a compact memory of past gradients, it resolves delayed feedback, reduces gradient variance, and delivers state‑of‑the‑art performance with minimal computational overhead. The work opens avenues for further research into ultra‑lightweight, parameter‑free adaptation mechanisms and more expressive feature‑space correction modules.