Let $G$ be an abelian group of bounded exponent and $A \subseteq G$. We show that if the collection of translates of $A$ has VC dimension at most $d$, then for every $ε>0$ there is a subgroup $H$ of $G$ of index at most $ε^{-d-o(1)}$ such that one can add or delete at most $ε|G|$ elements to/from $A$ to make it a union of $H$-cosets. We also establish a removal lemma with polynomial bounds, with applications to property testing, for induced bipartite patterns in a finite abelian group with bounded exponent.