Let $H(n, q^2)$ be a non-degenerate Hermitian variety of $PG(n, q^2)$, $n \ge 2$. Let NU$(n+1, q^2)$ be the graph whose vertices are the points of $PG(n, q^2) \setminus H(n, q^2)$ and two vertices $P_1$, $P_2$ are adjacent if the line joining $P_1$ and $P_2$ is tangent to $H(n, q^2)$. Then NU$(n + 1, q^2)$ is a strongly regular graph. In this paper we show that NU$(n + 1, q^2)$, $n \ne 3$, is not determined by its spectrum.