For $D$ being a subset of positive integers, the integer distance graph is the graph $G(D)$, whose vertex set is the set of integers, and edge set is the set of all pairs $uv$ with $|u-v| \in D$. It is known that $χ(G(D)) \leq |D|+1$. This article studies the problem (which is motivated by a conjecture of Zhu): "Is it true that $χ(G(D)) = |D|+1$ implies $ω(G(D)) \geq |D|+1$, where $ω(H)$ is the clique number of $H$?". We give a negative answer to this question, by showing an infinite class of integer distance graphs with $χ(G(D))=|D|+1$ but $ω(G(D))=|D|-1$.