We show that every nearly spherical manifold can be realized as the volume-preserving image of a round sphere, via the Brenier-McCann optimal transport map. This theorem extends Caffarelli's contraction theorem to nearly spherical manifolds and yields, as a corollary, a proof of a perturbative form of Milman's conjecture. The proof is based on a novel stability result for optimal transport maps on the sphere.