The main contribution of the current study is two-fold. First, we investigate the energy landscape of the Ising and Potts models on finite two-dimensional lattices without external fields in the low temperature regime. The complete analysis of the energy landscape of these models was unknown because of its complicated plateau saddle structure between the ground states. We characterize this structure completely in terms of a random walk on the set of sub-trees of a ladder graph. Second, we provide a considerable simplification of the well-known potential-theoretic approach to metastability. In particular, by replacing the role of variational principles such as the Dirichlet and Thomson principles with an $H^1$-approximation of the equilibrium potential, we develop a new method that can be applied to non-reversible dynamics as well in a simple manner. As an application of this method, we analyze metastable behavior of not only the reversible Metropolis-Hastings dynamics, but also of several interesting non-reversible dynamics associated with the low-temperature Ising and Potts models explained above, and derive the Eyring-Kramers law and the Markov chain model reduction of these models.