In this paper, some special distance spectral properties of graphs are considered. Concretely, we recursively construct an infinite family of trees with distance eigenvalue $-1$, and determine all $\{C_3,C_4\}$-free connected graphs with three distinct distance eigenvalues of which the smallest one is equal to $-3$, which partially answers a problem posed by Koolen, Hayat and Iqbal [Linear Algebra Appl. 505 (2016) 97--108]. Furthermore, we characterize all trees with three distinct distance eigenvalues.