In this paper, we seek to prove the equality of the $q$-graded fermionic sums conjectured by Hatayama et al. in its full generality, by extending the results of Di Francesco and Kedem to the non-simply laced case. To this end, we will derive explicit expressions for the quantum $Q$-system relations, which are quantum cluster mutations that correspond to the classical $Q$-system relations, and write the identity of the $q$-graded fermionic sums as a constant term identity. As an application, we will show that these quantum $Q$-system relations are consistent with the short exact sequence of the Feigin-Loktev fusion product of Kirillov-Reshetikhin modules obtained by Chari and Venkatesh.