We study the binary hypothesis testing problem where an adversary may potentially corrupt a fraction of the samples. The detector is, however, permitted to abstain from making a decision if (and only if) the adversary is present. We consider a few natural "contamination models" and characterize for them the trade-off between the error exponents of the four types of errors -- errors of deciding in favour of the incorrect hypothesis when the adversary is present and errors of abstaining or deciding in favour of the wrong hypothesis when the adversary is absent, under the two hypotheses.