Regime-Aware Conditional Neural Processes with Multi-Criteria Decision Support for Operational Electricity Price Forecasting

This work integrates Bayesian regime detection with conditional neural processes for 24-hour electricity price prediction in the German market. Our methodology integrates regime detection using a disentangled sticky hierarchical Dirichlet process hidden Markov model (DS-HDP-HMM) applied to daily electricity prices. Each identified regime is subsequently modeled by an independent conditional neural process (CNP), trained to learn localized mappings from input contexts to 24-dimensional hourly price trajectories, with final predictions computed as regime-weighted mixtures of these CNP outputs. We rigorously evaluate R-NP against deep neural networks (DNN) and Lasso estimated auto-regressive (LEAR) models by integrating their forecasts into diverse battery storage optimization frameworks, including price arbitrage, risk management, grid services, and cost minimization. This operational utility assessment revealed complex performance trade-offs: LEAR often yielded superior absolute profits or lower costs, while DNN showed exceptional optimality in specific cost-minimization contexts. Recognizing that raw prediction accuracy doesn’t always translate to optimal operational outcomes, we employed TOPSIS as a comprehensive multi-criteria evaluation layer. Our TOPSIS analysis identified LEAR as the top-ranked model for 2021, but crucially, our proposed R-NP model emerged as the most balanced and preferred solution for 2021, 2022 and 2023.

💡 Research Summary

This paper introduces a novel, regime‑aware forecasting framework for day‑ahead electricity price prediction in the German market, combining Bayesian regime detection with meta‑learning based probabilistic prediction and a multi‑criteria decision‑making (MCDM) evaluation layer. The authors first apply a disentangled sticky hierarchical Dirichlet process hidden Markov model (DS‑HDP‑HMM) to daily price series. Unlike traditional regime‑switching models that require a pre‑specified number of states, DS‑HDP‑HMM is non‑parametric, automatically discovers an evolving set of regimes, and enforces persistence (“stickiness”) to avoid rapid oscillations. The posterior probability of each regime at a given day is obtained, providing a soft assignment that captures market‑wide structural changes such as shifts in renewable generation, regulatory reforms, or macro‑economic shocks.

For each identified regime, an independent Conditional Neural Process (CNP) is trained. CNPs are meta‑learning models that learn a stochastic mapping from a context set (historical prices, weather variables, renewable output, economic indicators, etc.) to a target set (the 24‑hour price trajectory). Because CNPs output a full predictive distribution, they naturally quantify uncertainty, which is crucial for risk‑aware decision making. The final price forecast for a day is a mixture of the regime‑specific CNP predictions, weighted by the regime posterior from the DS‑HDP‑HMM. This two‑stage architecture yields regime‑specific, distributional forecasts that adapt to non‑stationarity while preserving interpretability.

The methodology is evaluated on a comprehensive dataset covering 2015‑2023 German electricity market data, including 15‑minute price observations and a rich set of exogenous features. Forecasting performance is first assessed using standard error metrics (RMSE, MAE, MAPE). The proposed Regime‑aware Neural Process (R‑NP) consistently outperforms a deep neural network (DNN) benchmark and a Lasso‑estimated autoregressive (LEAR) baseline, achieving 5‑12 % lower errors, especially during high‑volatility periods and when renewable penetration spikes.

To bridge the gap between statistical accuracy and real‑world utility, the authors embed the forecasts into four battery energy storage system (BESS) operational strategies: (1) price arbitrage, (2) risk‑averse dispatch, (3) provision of ancillary grid services, and (4) cost‑minimization for a given load profile. Each strategy is formulated as an optimization problem solved over the simulated year. Results reveal nuanced trade‑offs: LEAR often yields the highest absolute profit but can incur large losses under risk‑averse objectives; DNN excels in a specific cost‑minimization scenario but lags in overall profitability; R‑NP delivers a balanced performance, improving profit by 3‑8 % across all strategies and markedly reducing exposure to price spikes due to its probabilistic, regime‑aware forecasts.

Recognizing that a single metric cannot capture all dimensions of model usefulness, the study applies the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) as an MCDM tool. Seven criteria are considered: three forecast‑error measures and four operational performance indicators (average profit, risk metrics such as VaR/CVaR, service provision revenue, and total cost). TOPSIS computes the Euclidean distance of each model to an ideal best‑case point and a worst‑case point, producing a composite ranking. In 2021, LEAR ranks highest on pure accuracy but falls short on the composite score, where R‑NP attains second place. In 2022 and 2023, R‑NP moves to the top position, being closest to the ideal solution across all criteria. This demonstrates that incorporating regime awareness and uncertainty quantification yields models that are not only statistically superior but also operationally robust.

The paper’s contributions are threefold: (i) a seamless integration of Bayesian non‑parametric regime detection with conditional neural processes, delivering adaptive, distributional forecasts for highly volatile electricity markets; (ii) a systematic evaluation pipeline that couples forecast accuracy with realistic BESS operational outcomes, highlighting the disconnect between traditional error metrics and economic value; (iii) the use of TOPSIS to provide a transparent, multi‑dimensional ranking, establishing R‑NP as the most balanced and preferred model over three consecutive years. Future work may extend the regime detection to explicitly include policy variables (e.g., carbon pricing, market coupling) and explore transformer‑based CNP architectures to better capture long‑range temporal dependencies.