We study the approximability of general convex sets in $\mathbb{R}^n$ by intersections of halfspaces, where the approximation quality is measured with respect to the standard Gaussian distribution $N(0,I_n)$ and the complexity of an approximation is the number of halfspaces used. While a large body of research has considered the approximation of convex sets by intersections of halfspaces under distance metrics such as the Lebesgue measure and Hausdorff distance, prior to our work there has not been a systematic study of convex approximation under the Gaussian distribution. We establish a range of upper and lower bounds, both for general convex sets and for specific natural convex sets that are of particular interest. Our results demonstrate that the landscape of approximation is intriguingly different under the Gaussian distribution versus previously studied distance measures. For example, we show that $2^{Θ(\sqrt{n})}$ halfspaces are both necessary and sufficient to approximate the origin-centered $\ell_2$ ball of Gaussian volume 1/2 to any constant accuracy, and that for $1 \leq p < 2$, the origin-centered $\ell_p$ ball of Gaussian volume 1/2 can be approximated to any constant accuracy as an intersection of $2^{\widetilde{O}(n^{3/4})}$ many halfspaces. These bounds are quite different from known approximation results under more commonly studied distance measures. Our results are proved using techniques from many different areas. These include classical results on convex polyhedral approximation, Cramér-type bounds on large deviations from probability theory, and -- perhaps surprisingly -- a range of topics from computational complexity, including computational learning theory, unconditional pseudorandomness, and the study of influences and noise sensitivity in the analysis of Boolean functions.