Suppose $G$ is a finite group and $A\subseteq G$ is such that $\{gA:g\in G\}$ has VC-dimension strictly less than $k$. We find algebraically well-structured sets in $G$ which, up to a chosen $ε>0$, describe the structure of $A$ and behave regularly with respect to translates of $A$. For the subclass of groups with uniformly fixed finite exponent $r$, these algebraic objects are normal subgroups with index bounded in terms of $k$, $r$, and $ε$. For arbitrary groups, we use Bohr neighborhoods of bounded rank and width inside normal subgroups of bounded index. Our proofs are largely model theoretic, and heavily rely on a structural analysis of compactifications of pseudofinite groups as inverse limits of Lie groups. The introduction of Bohr neighborhoods into the nonabelian setting uses model theoretic methods related to the work of Breuillard, Green, and Tao and Hrushovski on approximate groups, as well as a result of Alekseev, Glebskii, and Gordon on approximate homomorphisms.