In this note we give a combinatorial and non-computational proof of the asymptotics of the integer moments of the moments of the characteristic polynomials of Haar distributed unitary matrices as the size of the matrix goes to infinity. This is achieved by relating these quantities to a lattice point count problem. Our main result is a new explicit expression for the leading order coefficient in the asymptotic as a volume of a certain region involving continuous Gelfand-Tsetlin patterns with constraints.