We consider the narrow wedge solution to the Kardar-Parisi-Zhang stochastic PDE under the characteristic $3:2:1$ scaling of time, space and fluctuations. We study the correlation of fluctuations at two different times. We show that when the times are close to each other, the correlation approaches one at a power-law rate with exponent $2/3$, while when the two times are remote from each other, the correlation tends to zero at a power-law rate with exponent $-1/3$. We also prove exponential-type tail bounds for differences of the solution at two space-time points. Three main tools are pivotal to proving these results: 1) a representation for the two-time distribution in terms of two independent narrow wedge solutions; 2) the Brownian Gibbs property of the KPZ line ensemble; and 3) recently proved one-point tail bounds on the narrow wedge solution.