Let $G$ be a connected graph with vertex set $V(G)$. The distance, $d_G(u,v)$, between vertices $u$ and $v$ in $G$ is defined as the length of a shortest path between $u$ and $v$ in $G$. The distance matrix of $G$ is the matrix $D(G)=(d_G(u,v))_{u,v\in V(G)}$. The second largest distance eigenvalue of $G$ is the second largest one in the spectrum of $D(G)$. We show that any connected graph with the second largest distance eigenvalue less than $\frac{-3+\sqrt{5}}{2}$ is chordal, and characterize those bicyclic graphs and split graphs with the second largest distance eigenvalue less than $-\frac{1}{2}$.