Tail-Aware Density Forecasting of Locally Explosive Time Series: A Neural Network Approach

This paper proposes a Mixture Density Network specifically designed for forecasting time series that exhibit locally explosive behavior. By incorporating skewed t-distributions as mixture components, our approach offers enhanced flexibility in capturing the skewed, heavy-tailed, and potentially multimodal nature of predictive densities associated with bubble dynamics modeled by mixed causal-noncausal ARMA processes. In addition, we implement an adaptive weighting scheme that emphasizes tail observations during training and hence leads to accurate density estimation in the extreme regions most relevant for financial applications. Equally important, once trained, the MDN produces near-instantaneous density forecasts. Through extensive Monte Carlo simulations and two empirical applications, on the natural gas price and inflation, we show that the proposed MDN-based framework delivers superior forecasting performance relative to existing approaches.

💡 Research Summary

The paper introduces a novel forecasting framework for locally explosive time‑series, such as asset‑price bubbles and sudden inflation spikes, by harnessing a specially designed Mixture Density Network (MDN). Traditional causal ARMA models fail in explosive regimes because forecasts revert to the unconditional mean, while mixed causal‑noncausal (MARMA) processes capture forward‑looking dynamics but lack closed‑form predictive densities. Existing Bayesian simulation or MCMC approaches are computationally heavy and struggle to estimate tail probabilities accurately.

To overcome these limitations, the authors replace the Gaussian kernels of the classic Bishop (1994) MDN with skew‑t distributions. A skew‑t component is parameterised by location (µ), scale (σ), skewness (ξ) and degrees‑of‑freedom (ν), allowing simultaneous control of asymmetry and heavy‑tailedness. This choice is crucial for modelling the multimodal, skewed, and fat‑tailed predictive densities that arise during bubble formation and collapse. The mixture comprises K = 10 components, providing sufficient flexibility without excessive complexity.

The network architecture is deliberately lightweight: an input vector of L past observations feeds a two‑layer fully‑connected perceptron (64 neurons per layer, ReLU activations). Five parallel output heads produce the mixture weights (π via softmax), µ (linear), σ and ν (softplus to enforce positivity), and ξ (linear). The total parameter count is roughly 8 000, which exceeds the sample size in the Monte‑Carlo and empirical studies but, according to recent “double‑descent” theory, this over‑parameterisation can improve generalisation.

A central methodological contribution is an adaptive weighting scheme that forces the network to focus on rare tail events during training. Tail observations are identified by a data‑driven box‑plot rule (lower bound ℓ, upper bound u). Each tail observation receives a weight w = q·T/|E| (with q the tail‑fraction), while non‑tail points keep weight 1. These weights are applied both in the mini‑batch sampling probability and as multiplicative factors in the negative log‑likelihood loss. Consequently, the effective contribution of tail points is amplified by a factor proportional to the inverse tail frequency, ensuring that the network learns accurate tail densities.

Because the training distribution is deliberately re‑weighted, the raw MDN output is biased toward over‑estimating extreme probabilities. The authors therefore employ a post‑hoc calibration step based on the Probability Integral Transform (PIT). Using a separate calibration set, they compute PIT values for each forecast, then train a series of XGBoost binary classifiers to estimate the conditional CDF of the PIT at a grid of thresholds τ∈