A Tolerance-Based Framework for Spatio-Temporal Forecast Validation Using the gamma-Index

Classical field forecast evaluation relies mainly on local scores such as RMSE or MAE. These metrics severely over-penalize small spatial or temporal displacements of coherent structures, a limitation known as the double-penalty issue and common to many forecasting domains. The present paper introduces a tolerance-based framework built on the three-dimensional gamma index, initially designed for medical dose verification, as a unified acceptance criterion for gridded forecasts. The method embeds explicit margins in space (DTA), time (TTA), and intensity (IDT), and evaluates whether predictions agree with observations within predefined physical bounds rather than through pixel-wise differences only. A synthetic illustration is first used to show why conventional metrics can misrepresent usable forecasts. The approach is then applied to satellite-derived SSI fields to demonstrate operational behaviour on a real dataset. Results confirm that the gamma criterion preserves structural consistency under minor positional noise while isolating physically significant discrepancies. The formulation is generic and can be implemented for any gridded variable provided meaningful tolerances are defined, offering a pragmatic complement to existing spatial verification tools in general forecasting workflows.

💡 Research Summary

The paper introduces a novel tolerance‑based verification framework for gridded forecasts by adapting the three‑dimensional γ‑index, originally developed for medical dose verification, to the spatio‑temporal domain of surface solar irradiance (SSI) forecasting. Traditional point‑wise error metrics such as RMSE, MAE, and MBE assume exact spatial and temporal coincidence between prediction and observation. This assumption leads to the well‑known “double‑penalty” problem: even modest displacements of coherent structures (e.g., a cloud shifted by a few kilometres or minutes) generate disproportionately large errors, potentially causing forecasters to reject otherwise usable predictions. Existing spatial verification approaches—neighbourhood/fuzzy methods, object‑based techniques, and multiscale analyses—mitigate the issue to some extent but lack a single, physically interpretable acceptance criterion expressed in the native units of space, time, and intensity.

The γ‑index addresses this gap by defining explicit tolerances: distance‑to‑agreement (DTA) in kilometres, time‑to‑agreement (TTA) in minutes, and intensity‑to‑agreement (IDT) in W m⁻². For each grid point (x, y, t) the index computes the normalized Euclidean distance in a three‑dimensional tolerance space between the forecast value and any observation within the DTA/TTA neighbourhood. Mathematically,

γ(x,y,t)=min_{(x′,y′)∈Ω_DTA, t′∈Ω_TTA} √