Performance of prior event rate ratio method in the presence of differential mortality or dropout

Purpose: Prior event rate ratio (PERR) method was proposed to control for measured or unmeasured confounders in real-world evaluation of effectiveness and safety of medical treatments using electronic medical records data. A widely cited simulation study showed that PERR estimate of treatment effect was biased in the presence of differential morality/dropout. However, the study only considered one specific PERR estimator of treatment effect and one specific scenario of differential mortality/dropout. To enhance understanding of the method, we replicated and extended the simulation to consider an alternative PERR estimator and multiple scenarios. Methods: Simulation studies were performed with varying rate of mortality/dropout, including the scenario in the previous study in which mortality/dropout was simultaneously influenced by treatment, confounder and prior event and scenarios that differed in the determinants of mortality/dropout. In addition to the PERR estimator used in the previous study (PERR_Prev) that involved data form both completers and non-completers, we also evaluated an alternative PERR estimator (PERR_Comp) that used data only from completers. Results: The bias of PERR_Prev in the previously considered mortality/dropout scenario was replicated. Bias of PERR_Comp was only about one-third in magnitude as compared to that of PERR_Prev in this scenario. Furthermore, PERR_Prev did but PERR_Comp did not give biased estimates of treatment effect in scenarios that mortality/dropout was influenced by treatment or confounder but not prior event. Conclusion: The PERR is better seen as a methodological framework within which there is more than one way to operationalize the estimation. Its performance depends on the specific operationalization. PERR_Comp provides unbiased estimates unless mortality/dropout is affected by prior event.

💡 Research Summary

The paper revisits the Prior Event Rate Ratio (PERR) method, a technique designed to control measured and unmeasured confounding in observational studies that use electronic medical records. A widely cited simulation (Uddin et al., 2015) had shown that the PERR estimator employed in that work—hereafter referred to as PERR_Prev—produced biased treatment‑effect estimates when differential mortality or dropout was jointly driven by treatment (X), a confounder (C), and the pre‑treatment event (Y₁). The authors argue that PERR is a methodological framework rather than a single formula, and that alternative operationalisations may behave differently under the same data‑generating mechanisms.

To test this hypothesis, they defined a second estimator, PERR_Comp, which uses only completers (subjects who survive or remain in the study after the post‑treatment period) in both the numerator and denominator of the ratio. They then constructed four simulation scenarios that vary which variables affect the dropout indicator M₂: (1) X + C + Y₁ (the original scenario), (2) C + Y₁, (3) C + X, and (4) C only. For each scenario, they varied the overall dropout rate from 0 % to 20 % (in 5 % increments) and generated 10 000 replicates of a dataset containing 100 000 individuals per replicate. The true causal effect of treatment on the post‑treatment binary outcome Y₂ was set to a risk ratio (RR) of 2.

Three estimators were evaluated in each simulation: (i) the original PERR_Prev, (ii) the new PERR_Comp, and (iii) the conventional unadjusted RR that ignores the pre‑treatment period. For each estimator they reported the mean estimate across replicates and the 2.5th/97.5th percentiles as a 95 % empirical interval.

Key findings:

• In scenario 1, PERR_Prev showed increasingly negative bias as dropout rose, reaching an average estimate of 1.83 at 20 % dropout (true RR = 2). PERR_Comp was essentially unbiased up to 10 % dropout and only modestly over‑estimated (2.05) at 20 % dropout—its absolute bias was roughly one‑third that of PERR_Prev.
• Scenario 2 (C + Y₁) produced a similar pattern but with milder bias; at 20 % dropout PERR_Comp averaged 2.02, PERR_Prev 1.91, while the unadjusted RR was markedly inflated (2.55) because of residual confounding.
• In scenarios 3 and 4, where dropout was not a function of the pre‑treatment event, PERR_Comp yielded accurate estimates across all dropout levels, whereas PERR_Prev continued to display bias. The unadjusted RR remained biased due to confounding, though its bias was partially offset by the differential dropout effect as dropout increased.

The authors interpret these results as evidence that the choice of PERR estimator critically determines robustness to differential dropout. When dropout is independent of the pre‑treatment event, the completer‑only estimator (PERR_Comp) is essentially unbiased, even if dropout depends on treatment or the confounder. Bias only emerges when the pre‑treatment event itself influences dropout, but even then the magnitude is substantially smaller than that observed with PERR_Prev.

The discussion emphasizes that PERR should be viewed as a flexible framework; researchers must consider the causal pathways linking treatment, confounders, prior events, and loss‑to‑follow‑up before selecting an estimator. The paper also notes that while the simulations focused on binary outcomes, analogous investigations are needed for time‑to‑event outcomes.

In conclusion, differential mortality or dropout does not automatically invalidate the PERR approach. Its impact depends on the specific operationalisation of the estimator. The completer‑only version (PERR_Comp) provides reliable treatment‑effect estimates unless the dropout mechanism is directly driven by the prior event, in which case some bias remains but is markedly less severe than with the mixed‑population estimator. This insight refines guidance for applying PERR in real‑world evidence studies and underscores the importance of pre‑analysis checks on dropout mechanisms.