Abstract:Empirical causal claims depend on many analyst decisions, from selecting covariates to choosing estimators. Existing robustness tools summarize how results vary across these choices, but, to the best of our knowledge, do not answer: \textbf{How many analyst decisions must change to reach a specification, which is a set of choices, whose confidence interval (CI) contains zero?} We introduce \emph{Minimum Specification Perturbation (MSP)}, the smallest number of changes. MSP is small under the null, grows with effect strength and captures distance-to-falsification information that dispersion-based summaries cannot report; when making decisions under weak effects, an MSP-based rule yields lower false-positive rates than dispersion-based rules. We show that Fragility Index and MSP measure orthogonal vulnerabilities: fragility to influential observations need not imply fragility to specification choices. On the LaLonde benchmark, MSP = 1 implies that one decision change makes the CI contain zero. We further provide exact permutation calibration under randomization and characterize computation, showing tractable cases under additive structure and NP-hardness in general.
| Comments: | 36 pages, 2 figures |
| Subjects: | Methodology (stat.ME); Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.01579 [stat.ME] |
| (or arXiv:2605.01579v1 [stat.ME] for this version) | |
| https://doi.org/10.48550/arXiv.2605.01579 arXiv-issued DOI via DataCite (pending registration) |
From: Hoang Dang Van Cong [view email]
[v1]
Sat, 2 May 2026 19:04:08 UTC (81 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。