Efficient Variance-reduced Estimation from Generative EHR Models: The SCOPE and REACH Estimators

Generative models trained using self-supervision of tokenized electronic health record (EHR) timelines show promise for clinical outcome prediction. This is typically done using Monte Carlo simulation for future patient trajectories. However, existing approaches suffer from three key limitations: sparse estimate distributions that poorly differentiate patient risk levels, extreme computational costs, and high sampling variance. We propose two new estimators: the Sum of Conditional Outcome Probability Estimator (SCOPE) and Risk Estimation from Anticipated Conditional Hazards (REACH), that leverage next-token probability distributions discarded by standard Monte Carlo. We prove both estimators are unbiased and that REACH guarantees variance reduction over Monte Carlo sampling for any model and outcome. Empirically, on hospital mortality prediction in MIMIC-IV using the ETHOS-ARES framework, SCOPE and REACH match 100-sample Monte Carlo performance using only 10-11 samples (95% CI: [9,11]), representing a ~10x reduction in inference cost without degrading calibration. For ICU admission prediction, efficiency gains are more modest (~1.2x), which we attribute to the outcome’s lower “spontaneity,” a property we characterize theoretically and empirically. These methods substantially improve the feasibility of deploying generative EHR models in resource-constrained clinical settings.

💡 Research Summary

This paper addresses three major shortcomings of the Monte‑Carlo (MC) approach for outcome risk estimation with generative electronic health record (EHR) models: (1) sparse estimate distributions that limit discrimination, especially for rare events; (2) prohibitive computational cost due to the need for many simulated future trajectories; and (3) high sampling variance inherent in binary outcome sampling. The authors introduce two novel estimators—SCOPE (Sum of Conditional Outcome Probability Estimates) and REACH (Risk Estimation from Anticipated Conditional Hazards)—that exploit the next‑token probability distribution which standard MC discards.

SCOPE computes, for each simulated trajectory, the sum of the model‑provided conditional probabilities that the next token is the outcome of interest (O) until O appears or a predefined horizon T_E is reached. The estimator is the average of these sums across n trajectories: S = (1/n) Σ_i Σ_{t≤min(T_E,T_O)} P(O | history). This yields a continuous risk score rather than the discrete 0,1/n,… values of MC, thereby alleviating sparsity. The authors note that, in theory, the sum can exceed 1; however, empirical results on 43,047 patients with 100 trajectories each never produced values above 1, and clipping would break unbiasedness.

REACH takes a more radical step: it generates “outcome‑free” trajectories by setting the probability of O to zero in the model’s next‑token distribution (denoted \hat{P}). For each step t, the model’s conditional probability h_t = P(O | history) is interpreted as a hazard; the patient survives to the next step with probability 1‑h_t. The risk estimate for a trajectory is 1‑∏_{t=1}^{T_E} (1‑h_t), and the final estimator is the average over n such trajectories: R = (1/n) Σ_i