In this paper, we study the combinatorics of congruence subgroups of the modular group by generalizing results obtained in the non-modular case. For this, we define a notion of irreducible solutions from which we can build all the solutions. In particular, we give a particular solution, irreducible for any $N$, and the list of irreducible solutions for $N \leq 6$.