We define a natural notion of higher order stability and show that subsets of $\mathbb{F}_p^n$ that are tame in this sense can be approximately described by a union of low-complexity quadratic varieties, up to linear error. This generalizes the arithmetic regularity lemma for stable subsets of $\mathbb{F}_p^n$, proved in earlier work of the authors, to the realm of higher-order Fourier analysis. This result is strictly stronger than the structure theorem for sets of bounded $\mathrm{VC}_2$-dimension, first proved by the authors in earlier versions of this paper and now available as a separate manuscript arXiv:2510.12867. Taken together, these results provide group theoretic analogues of results obtained for 3-uniform hypergraphs in arXiv:2111.01737.