A result of Pyber states that every finite group $G$ contains an abelian subgroup whose order is quasi-polynomially large in $\lvert G\rvert$. We prove a similar result for $K$-approximate subgroups of solvable groups under only modest restrictions on $K$. We show that, if $A$ is a finite $K$-approximate group contained in some solvable group, then some abelian group intersects $A^4$ in at least $\exp(Ω(\log^{1/6}\lvert A\rvert/\log 2K))$ elements. We also prove a similar result for approximate subgroups of finite groups with no large alternating subquotients. Along the way, we obtain polynomial (instead of quasi-polynomial) bounds for the same statement of approximate subgroups of linear groups. We give two applications. Firstly, we consider the conjecture of Alon that every finite group $G$ admits a Cayley graph with clique number and independence number $O(\log\lvert G\rvert)$. Conlon, Fox, Pham, and Yepremyan have recently proven that, for almost all positive integers $N$, every abelian group of order $N$ satisfies Alon's conjecture. Extending their result, we verify Alon's conjecture for all (not necessarily abelian) groups of almost all orders. Secondly, we prove a "local" version of Roth's theorem in (many) non-abelian settings with quasi-polynomial bounds, using the recent breakthroughs of Kelley and Meka on Roth's theorem and of Jaber, Liu, Lovett, Ostuni, and Sawhney on the corners problem.