We consider directed last passage percolation on $\mathbb{Z}^2$ with exponential passage times on the vertices. A topic of great interest is the coupling structure of the weights of geodesics as the endpoints are varied spatially and temporally. A particular specialization is when one considers geodesics to points varying in the time direction starting from a given initial data. This paper considers the flat initial condition which corresponds to line-to-point last passage times. Settling a conjecture by Ferrari and Spohn (SIGMA, 2016), we show that for the passage times from the line $x+y=0$ to the points $(r,r)$ and $(n,n)$, denoted $X_{r}$ and $X_{n}$ respectively, as $n\to \infty$ and $\frac{r}{n}$ is small but bounded away from zero, the covariance satisfies $$\mbox{Cov}(X_{r},X_{n})=Θ\left((\frac{r}{n})^{4/3+o(1)} n^{2/3}\right),$$ thereby establishing $\frac{4}{3}$ as the temporal covariance exponent. This differs from the corresponding exponent for the droplet initial condition recently rigorously established in Ferrari and Occelli (2018), Basu and Ganguly (2018), and requires novel arguments. Key ingredients include the understanding of geodesic geometry and recent advances in quantitative comparison of geodesic weight profiles to Brownian motion using the Brownian Gibbs property. The proof methods are expected to be applicable for a wider class of initial data.