Let $G$ be a finite classical group generated by transvections, i.e., one of $\operatorname{SL}_n(q)$, $\operatorname{SU}_n(q)$, $\operatorname{Sp}_{2n}(q)$, or $\operatorname{O}^\pm_{2n}(q)$ ($q$ even), and let $X$ be a generating set for $G$ containing at least one transvection. Building on work of Garonzi, Halasi, and Somlai, we prove that the diameter of the Cayley graph $\operatorname{Cay}(G, X)$ is bounded by $(n \log q)^C$ for some constant $C$. This confirms Babai's conjecture on the diameter of finite simple groups in the case of generating sets containing a transvection. By combining this with a result of the author and Jezernik it follows that if $G$ is one of $\operatorname{SL}_n(q)$, $\operatorname{SU}_n(q)$, $\operatorname{Sp}_{2n}(q)$ and $X$ contains three random generators then with high probability the diameter $\operatorname{Cay}(G, X)$ is bounded by $n^{O(\log q)}$. This confirms Babai's conjecture for non-orthogonal classical simple groups over small fields and three random generators.