A Survey on Uncertainty Quantification Methods for Deep Learning

Deep neural networks (DNNs) have achieved tremendous success in computer vision, natural language processing, and scientific and engineering domains. However, DNNs can make unexpected, incorrect, yet overconfident predictions, leading to serious consequences in high-stakes applications such as autonomous driving, medical diagnosis, and disaster response. Uncertainty quantification (UQ) estimates the confidence of DNN predictions in addition to their accuracy. In recent years, many UQ methods have been developed for DNNs. It is valuable to systematically categorize these methods and compare their strengths and limitations. Existing surveys mostly categorize UQ methodologies by neural network architecture or Bayesian formulation, while overlooking the uncertainty sources each method addresses, making it difficult to select an appropriate approach in practice. To fill this gap, this paper presents a taxonomy of UQ methods for DNNs based on uncertainty sources (e.g., data versus model uncertainty). We summarize the advantages and disadvantages of each category, and illustrate how UQ can be applied to machine learning problems (e.g., active learning, out-of-distribution robustness, and deep reinforcement learning). We also identify future research directions, including UQ for large language models (LLMs), AI-driven scientific simulations, and deep neural networks with structured outputs.

💡 Research Summary

The paper “A Survey on Uncertainty Quantification Methods for Deep Learning” provides a comprehensive review of techniques designed to estimate the confidence of predictions made by deep neural networks (DNNs). It begins by highlighting the critical need for uncertainty quantification (UQ) in high-stakes applications like autonomous driving and medical diagnosis, where overconfident yet incorrect predictions can have severe consequences.

The survey’s primary contribution is a novel taxonomy of UQ methods based on the source of uncertainty they address, moving beyond existing categorizations based solely on network architecture (e.g., Bayesian Neural Networks) or methodological perspective (frequentist vs. Bayesian). It clearly distinguishes between two fundamental types: Epistemic (model) uncertainty, which stems from a lack of knowledge due to limited data, imperfect models, or out-of-distribution inputs and is reducible with more information; and Aleatoric (data) uncertainty, which arises from inherent noise, ambiguity, or class overlap in the data itself and is irreducible.

The core of the paper (Section 4) systematically organizes UQ methods into three categories: 1) those capturing primarily data uncertainty (e.g., heteroscedastic noise models, Evidential Deep Learning), 2) those capturing primarily model uncertainty (e.g., Deep Ensembles, Monte Carlo Dropout, Bayesian Neural Networks), and 3) hybrid methods aiming to capture both (e.g., Bayesian Ensembles). For each category and prominent method within, the authors detail the underlying principles, advantages, and practical limitations, offering valuable guidance for method selection.

Furthermore, the paper connects UQ to essential machine learning paradigms, demonstrating its role in Out-of-Distribution (OOD) detection, Active Learning (for efficient data labeling), and safe Deep Reinforcement Learning. It also discusses practical application domains such as medical imaging and geosciences.

Finally, the survey identifies promising future research directions. These include developing UQ for Large Language Models (LLMs) to mitigate hallucination, integrating UQ into AI-driven scientific simulations for trustworthy discovery, quantifying uncertainty in DNNs with structured outputs (e.g., for spatiotemporal or graph data), and combining UQ with explainable AI (XAI) techniques. By providing this source-centric taxonomy and broad perspective, the paper serves as a valuable resource for researchers and practitioners aiming to build more reliable and trustworthy deep learning systems.