As a fundamental metric for quantifying quantum advantage in non-local games, the quantum chromatic number reveals the power of entanglement in distributed tasks. In this paper, we investigate this parameter for $q$-ary Hamming graphs and a generalization of Hadamard graphs. Our main results establish an exponential separation between the quantum and classical chromatic numbers for both graph families, and determine the exact quantum chromatic numbers in several regimes. Our analysis builds on known upper and lower bounds via modulus-one orthogonal representations and minimum eigenvalues, respectively. Previous results for Hamming graphs $H(n,q,d)$ were restricted to specific cases: the minimum eigenvalue was only identified for $d > (q-1)n/q$, while modulus-one orthogonal representations had only been constructed for the binary case ($q=2$) with $d \ge n/2$. In this work, we fill several gaps in the existing literature by developing a linear programming approach to construct modulus-one orthogonal representations for arbitrary relative distances, and using the trace method to determine the minimum eigenvalues in the regime where $d$ lies slightly below the threshold $(q-1)n/q$. For generalized Hadamard graphs over cyclic groups and finite fields, by determining their minimum eigenvalues, we show that the spectral lower bound matches the natural upper bound on the quantum chromatic number. On the classical side, we apply the method of forbidden intersection pattern of Frankl and Rödl to obtain an exponential lower bound on the chromatic number, thereby quantifying the separation between the quantum and classical quantities.