The hybrid confirmation tree: A robust strategy for hybrid intelligence

Combining human and artificial intelligence (AI) is a potentially powerful approach to boost decision accuracy. However, few such approaches exist that effectively integrate both types of intelligence while maintaining human agency. Here, we introduce and evaluate the hybrid confirmation tree, a simple aggregation strategy that compares the independent decisions of both a human and AI, with disagreements triggering a second human tiebreaker. Through analytical derivations, we show that the hybrid confirmation tree can match and exceed the accuracy of a three-person human majority vote while requiring fewer human inputs, particularly when AI accuracy is comparable to or exceeds human accuracy. We analytically demonstrate that the hybrid confirmation tree’s ability to achieve complementarity – outperforming individual humans, AI, and the majority vote – is maximized when human and AI accuracies are similar and their decisions are not overly correlated. Empirical reanalysis of six real-world datasets (covering skin cancer diagnosis, deepfake detection, geopolitical forecasting, and criminal rearrest) validates these findings, showing that the hybrid confirmation tree improves accuracy over the majority vote by up to 10 percentage points while reducing the cost of decision making by 28–44$%$. Furthermore, the hybrid confirmation tree provides greater flexibility in navigating true and false positive trade-offs compared to fixed human-only heuristics like hierarchies and polyarchies. The hybrid confirmation tree emerges as a practical, efficient, and robust strategy for hybrid collective intelligence that maintains human agency.

💡 Research Summary

**
The paper introduces the Hybrid Confirmation Tree (HCT), a simple yet powerful aggregation scheme for combining independent decisions from a human and an artificial intelligence (AI) system. The procedure works as follows: a first human makes a judgment, an AI makes an independent judgment, and if the two agree the decision is accepted immediately. If they disagree, a second human steps in to break the tie, guaranteeing that the final outcome is always approved by at least one human. This design preserves human agency while exploiting the speed and, potentially, higher accuracy of AI.

The authors first situate HCT within the broader literature on crowd wisdom and organizational decision structures. Traditional majority voting (three‑person human majority) can improve accuracy when individual accuracies exceed chance and errors are uncorrelated, but real‑world settings often violate independence, leading to correlated mistakes that diminish the benefit. Hierarchical (unanimous) and polyarchic (single‑approval) structures address false‑positive/false‑negative trade‑offs but are either overly conservative or overly permissive. HCT offers a middle ground: it leverages AI as a low‑cost “first filter” while retaining a human‑only final check.

Analytically, the authors model human accuracy (p_h), AI accuracy (p_a), and the correlation (\rho) between their binary decisions. They derive closed‑form expressions for the expected accuracy of HCT and for a three‑person human majority vote. The key theoretical results are: (1) When (p_a > p_h), HCT outperforms the human majority; when (p_a < p_h), the reverse holds. (2) If (p_a = p_h), both methods have identical accuracy, but HCT requires on average only one human judgment per case instead of three, cutting human effort by roughly two‑thirds. (3) The benefit of HCT grows as (\rho) decreases; low correlation means the human and AI make different kinds of errors, allowing error‑cancellation. Simulations across the full (