The normalized distance Laplacian matrix $\mathcal{D}^{\mathcal{L}}(G)$ of a graph $G$ is a natural generalization of the normalized Laplacian matrix, arising from the matrix of pairwise distances between vertices rather than the adjacency matrix. Following the motif that this matrix behaves quite differently to the normalized Laplacian matrix, we show that both the spectral gap and Cheeger constant of $\mathcal{D}^{\mathcal{L}}(G)$ are bounded away from $0$ independently of the graph $G$. The spectral result holds more generally for finite metric spaces.