By examining the deterministic limit of a general $ε$-dependent generator for Markovian dynamics, which includes the continuous Fokker-Planck equations and discrete chemical master equations as two special cases, the intrinsic connections among mesoscopic stochastic dynamics, deterministic ODEs or PDEs, large deviations rate function, and macroscopic thermodynamic potential are established. Our result not only solves the long-lasting question on the origin of entropy function in classical irreversible thermodynamics, but also reveals an emergent feature that arises automatically during the deterministic limit, through its large deviations rate function, with both time-reversible dynamics equipped with a Hamiltonian function and time-irreversible dynamics equipped with an entropy function.