For a graph invariant $π$, the Contraction($π$) problem consists in, given a graph $G$ and two positive integers $k,d$, deciding whether one can contract at most $k$ edges of $G$ to obtain a graph in which $π$ has dropped by at least $d$. Galby et al. [ISAAC 2019, MFCS 2019] recently studied the case where $π$ is the size of a minimum dominating set. We focus on graph invariants defined as the minimum size of a vertex set that hits all the occurrences of graphs in a collection ${\cal H}$ according to a fixed containment relation. We prove co-NP-hardness results under some assumptions on the graphs in ${\cal H}$, which in particular imply that Contraction($π$) is co-NP-hard even for fixed $k=d=1$ when $π$ is the size of a minimum feedback vertex set or an odd cycle transversal. In sharp contrast, we show that when $π$ is the size of a minimum vertex cover, the problem is in XP parameterized by $d$.