A pseudoforest is a graph whose connected components have at most one cycle. Let X be a pseudoforest modulator of graph G, i. e. a vertex subset of G such that G-X is a pseudoforest. We show that Vertex Cover admits a polynomial kernel being parameterized by the size of the pseudoforest modulator. In other words, we provide a polynomial time algorithm that for an input graph G and integer k, outputs a graph G' and integer k', such that G' has O(|X|12) vertices and G has a vertex cover of size k if and only if G' has vertex cover of size k'. We complement our findings by proving that there is no polynomial kernel for Vertex Cover parameterized by the size of a modulator to a mock forest (a graph where no cycles share a vertex) unless NP is a subset of coNP/poly. In particular, this also rules out polynomial kernels when parameterized by the size of a modulator to outerplanar and cactus graphs.