In this work, we study the eccentricity spectra of zero divisor graphs (ZDGs) associated with the ring $\mathbb{Z}_n.$ While previous studies have examined the Laplacian and distance Laplacian spectra of ZDGs, the eccentricity spectra have remained largely unknown due to the unique features of the eccentricity matrix. More specifically, we prove that for a prime $p$, the ZDG and extended ZDG of $\mathbb{Z}_{p^t}$ have integral eccentricity eigenvalues for $t \geq 3$ and $t \geq 2$, respectively. We also find the eccentricity spectra for specific classes of ZDGs and the relationship between the eccentricity matrix of these ZDGs and their tree structures using matrix analysis tools. In addition, for the usefulness of the energy gap in applications, we have calculated the eccentricity energy gap of ZDGs. These findings reveal interesting behaviours of the eccentricity matrix and may contribute to a more profound understanding of the structural properties of ZDGs.