In their study of infinite flag varieties, Lam, Lee, and Shimozono (2021) introduced bumpless pipe dreams in a new combinatorial formula for double Schubert polynomials. These polynomials are the TxT-equivariant cohomology classes of matrix Schubert varieties and of their flat degenerations. We give diagonal term orders with respect to which bumpless pipe dreams index the irreducible components of diagonal Gröbner degenerations of matrix Schubert varieties, counted with scheme-theoretic multiplicity. This indexing was conjectured by Hamaker, Pechenik, and Weigandt (2022). This result establishes that bumpless pipe dreams are dual to and as geometrically natural as classical pipe dreams, for which an analogous anti-diagonal theory was developed by Knutson and Miller (2005).