Doing More With Less: Mismatch-Based Risk-Limiting Audits

One approach to risk-limiting audits (RLAs) compares randomly selected cast vote records (CVRs) to votes read by human auditors from the corresponding ballot cards. Historically, such methods reduce audit sample sizes by considering how each sampled CVR differs from the corresponding true vote, not merely whether they differ. Here we investigate the latter approach, auditing by testing whether the total number of mismatches in the full set of CVRs exceeds the minimum number of CVR errors required for the reported outcome to be wrong (the “CVR margin”). This strategy makes it possible to audit more social choice functions and simplifies RLAs conceptually, which makes it easier to explain than some other RLA approaches. The cost is larger sample sizes. “Mismatch-based RLAs” only require a lower bound on the CVR margin, which for some social choice functions is easier to calculate than the effect of particular errors. When the population rate of mismatches is low and the lower bound on the CVR margin is close to the true CVR margin, the increase in sample size is small. However, the increase may be very large when errors include errors that, if corrected, would widen the CVR margin rather than narrow it; errors affect the margin between candidates other than the reported winner with the fewest votes and the reported loser with the most votes; or errors that affect different margins.

💡 Research Summary

The paper introduces a “mismatch‑based” risk‑limiting audit (RLA) that tests whether the total number of mismatches between cast‑vote records (CVRs) and the actual votes on ballot cards exceeds the minimum number of CVR errors required to overturn the reported election outcome – the so‑called CVR margin. Unlike traditional card‑level comparison audits, which exploit the exact nature of each sampled discrepancy to keep sample sizes small, the mismatch‑based approach treats every possible error as if it were the worst‑case error for the margin. Consequently, it only needs a lower bound on the CVR margin (V⁻) rather than the exact margin, making it applicable to a broader class of social‑choice functions, especially those for which computing the exact CVR margin is computationally hard (e.g., Instant‑Runoff Voting, multi‑seat Single Transferable Vote).

The authors formalize the CVR margin as the smallest Hamming‑distance radius around the CVR vector C that contains a vote profile B producing a different election outcome. The normalized margin v = V(C)/N (where N is the number of ballots) is the proportion of CVRs that must be altered to change the result. In practice, auditors can usually compute a tractable lower bound V⁻ ≤ V(C); the corresponding proportion v⁻ = V⁻/N is used in the audit.

The audit proceeds by testing the null hypothesis H₀: M ≥ V⁻, where M is the total number of observed mismatches in the full CVR set, equivalently m ≥ v⁻ for the mismatch rate. To conduct this test, the paper leverages the SHANGRLA framework, which represents correctness of an election outcome via a collection of bounded, non‑negative “assorters”. For each assorter A, the true mean over the ballot votes must exceed ½ if the reported outcome is correct. In a standard comparison audit, the reference values x_i are set to A(c_i), the assorter value of the CVR, allowing a precise assessment of over‑ or under‑statements. In the mismatch‑based audit, the reference values are reduced to a binary indicator of whether the CVR matches the true vote, simplifying the test but making it pessimistic: any mismatch is treated as a potential overstatement that could narrow the margin.

Statistically, the audit constructs an “overstatement assorter” B(b_i) = (u + A(b_i) – x_i)/(2u – ν), where u is an upper bound on A and ν is twice the margin of the reference vector. The hypothesis H₀ translates to checking whether the sample mean of B exceeds ½, which can be done with sequential sampling methods (e.g., Wald’s SPRT) at a pre‑specified risk limit α (commonly 5%).

The paper evaluates the method through simulations of two election types: (1) a two‑candidate plurality contest and (2) an Instant‑Runoff Voting (IRV) contest. Various error patterns are examined: errors that shrink the margin (e.g., turning a winner vote into a loser vote), errors that expand the margin (e.g., turning an invalid vote into a winner vote), and errors that affect margins between non‑top candidates or across multiple rounds. The findings are nuanced:

• When the true mismatch rate is very low (≈0.5 %) and the lower bound V⁻ is close to the true CVR margin V, the required sample size for the mismatch‑based audit is comparable to that of a finely tuned comparison audit.
• If errors predominantly widen the margin, the mismatch‑based audit can actually require fewer samples than a comparison audit because the observed mismatches provide “positive evidence” for the reported outcome.
• Conversely, when errors are of the worst‑case type—especially those that would change the winner in a later round of an IRV or STV election—the required sample size can balloon, sometimes by an order of magnitude or more. This is especially pronounced when the lower bound V⁻ is loose, inflating the conservatism of the test.

A critical practical limitation is the need for a one‑to‑one linkage between each CVR and its physical ballot card. The method is therefore only feasible in jurisdictions that use central‑count optical‑scan systems or are willing to rescan precinct‑tabulated cards to generate linked CVRs. Batch‑level or ballot‑polling audits cannot be performed with this approach.

Despite the potential for larger samples, the mismatch‑based RLA offers a universal auditing tool for any election where a CVR margin lower bound can be computed. For complex social‑choice functions such as multi‑seat STV—where no efficient comparison audit is known—the mismatch‑based audit may be the only practical RLA. The authors argue that this contributes to the broader “evidence‑based elections” movement by providing a statistically rigorous, conceptually simple audit that can be explained to non‑technical stakeholders.

The paper concludes with several avenues for future work: (1) developing tighter algorithms for computing CVR‑margin lower bounds, (2) designing assorter families tailored to multi‑round voting systems to reduce conservatism, (3) improving the cryptographic or procedural mechanisms that guarantee CVR‑ballot linkage, and (4) conducting field pilots to assess operational costs and public perception. In sum, the work expands the toolkit of risk‑limiting audits, balancing universality against sample‑size efficiency, and highlights the trade‑offs election officials must consider when adopting RLAs for diverse voting systems.