We consider the problem of counting matrices over a finite field with fixed rank and support contained in a fixed set. The count of such matrices gives a $q$-analogue of the classical rook and hit numbers, known as the $q$-rook and $q$-hit numbers. They are known not to be polynomial in $q$ in general. We use inclusion-exclusion on the support of the matrices and the orbit counting method of Lewis et al. to show that the residues of these functions in low degrees are polynomial. We define a generalization of the classical rook and hit numbers which count placements of certain classes of graphs. These give us a formula for residues of the $q$-rook and $q$-hit numbers in low degrees. We analyze the residues of the $q$-hit number and show that the coefficient of $q-1$ in the $q$-hit number is always non-negative.