Improved Inference for CSDID Using the Cluster Jackknife

Obtaining reliable inferences with traditional difference-in-differences (DiD) methods can be difficult. Problems can arise when both outcomes and errors are serially correlated, when there are few clusters or few treated clusters, when cluster sizes vary greatly, and in various other cases. In recent years, recognition of the ``staggered adoption’’ problem has shifted the focus away from inference towards consistent estimation of treatment effects. One of the most popular new estimators is the CSDID procedure of Callaway and Sant’Anna (2021). We find that the issues of over-rejection with few clusters and/or few treated clusters are at least as severe for CSDID as for traditional DiD methods. We also propose using a cluster jackknife for inference with CSDID, which simulations suggest greatly improves inference. We provide software packages in Stata csdidjack and R didjack to calculate cluster-jackknife standard errors easily.

💡 Research Summary

This paper investigates the inference problems that arise when applying the recent Callaway‑Sant’Anna (2021) CSDID estimator, especially in settings with few clusters, few treated clusters, serial correlation, or highly unbalanced cluster sizes. While CSDID resolves the bias issues associated with staggered adoption that plagued the traditional two‑way fixed‑effects (TWFE) DiD approach, the authors demonstrate through both theoretical discussion and extensive Monte‑Carlo simulations that the conventional cluster‑robust standard errors and the wild‑cluster bootstrap continue to suffer from severe over‑rejection when the number of clusters is modest. To address this, they propose a cluster jackknife procedure tailored to CSDID. The jackknife systematically leaves out each cluster, recomputes the CSDID ATT estimates, and uses the variability across these leave‑one‑out estimates to construct standard errors. This approach retains the ability to accommodate within‑cluster correlation without relying on large‑cluster asymptotics. Simulation results across a variety of data‑generating processes—including heterogeneous treatment effects, varying panel balance, and extreme cluster‑size heterogeneity—show that the jackknife standard errors achieve nominal 5 % rejection rates and substantially lower mean‑squared error than both the conventional cluster‑robust and wild‑bootstrap methods. The improvement is especially pronounced when the number of treated clusters is very small (e.g., one or two), although the authors acknowledge that the “few treated clusters” problem is not completely eliminated. To facilitate adoption, the authors release two software packages: csdidjack for Stata and didjack for R, which implement the jackknife procedure with a few lines of code. They also illustrate the practical benefits with two empirical applications—one on a Canadian health policy reform and another on a European regional development program—showing tighter confidence intervals and more credible inference. In sum, the paper provides a concrete, easy‑to‑implement solution that markedly enhances the reliability of inference with the CSDID estimator, thereby strengthening causal claims in modern DiD research.