We study the structures of ordinary simple Hurwitz numbers and monotone Hurwitz numbers with varying genus. More precisely, we prove that when the ramification type is fixed and the genus is treated as a variable, the connected monotone Hurwitz number is a linear combination of products of exponentials and polynomials, and the ordinary simple Hurwitz number is a linear combination of exponentials. Using these structural properties, we also derive the large genus asymptotics of these two kinds of Hurwitz numbers. As a result, we prove one conjecture, and disprove another, both proposed by Do, He and Robertson.