An integer linear system is a set of inequalities with integer constraints. The solution graph of an integer linear system is an undirected graph defined on the set of feasible solutions to the integer linear system. In this graph, a pair of feasible solutions is connected by an edge if the Hamming distance between them is one. In this paper, we consider a condition under which the solution graph is connected for any right-hand side vector. First, we prove that if the solution graph is connected for any right-hand side vector, then the coefficient matrix of the system does not contain some forbidden pattern as a submatrix. Next, we prove that if at least one of (i) the number of rows is at most 3, (ii) the number of columns is at most 2, (iii) the number of rows is 4 and the number of columns is 3 holds, then the condition that the coefficient matrix of the system does not contain the forbidden pattern is a sufficient condition under which the solution graph is connected for any right-hand side vector. This result is stronger than a known necessary and sufficient condition since the set of coefficient matrix dimensions is strictly larger.