In this article we investigate the $N$-point min-max and the max-min polarization problems on the sphere for a large class of potentials in $\mathbb{R}^n$. We derive universal lower and upper bounds on the polarization of spherical designs of fixed dimension, strength, and cardinality. The bounds are universal in the sense that they are a convex combination of potential function evaluations with nodes and weights independent of the class of potentials. As a consequence of our lower bounds, we obtain the Fazekas-Levenshtein bounds on the covering radius of spherical designs. Utilizing the existence of spherical designs, our polarization bounds are extended to general configurations. As examples we completely solve the min-max polarization problem for $120$ points on $\mathbb{S}^3$ and show that the $600$-cell is universally optimal for that problem. We also provide alternative methods for solving the max-min polarization problem when the number of points $N$ does not exceed the dimension $n$ and when $N=n+1$. We further show that the cross-polytope has the best max-min polarization constant among all spherical $2$-designs of $N=2n$ points for $n=2,3,4$; for $n\geq 5$, this statement is conditional on a well-known conjecture that the cross-polytope has the best covering radius. This max-min optimality is also established for all so-called centered codes.