The emergence of the exit events from a bounded domain containing a stable fixed point induced by non-Gaussian Lévy fluctuations plays a pivotal role in practical physical systems. In the limit of weak noise, we develop a Hamiltonian formalism under the Lévy fluctuations with exponentially light jumps for one- and two-dimensional stochastic dynamical systems. This formalism is based on a recently proved large deviation principle for dynamical systems under non-Gaussian Lévy perturbations. We demonstrate how to compute the most probable exit path and the quasi-potential by several examples. Meanwhile, we explore the impacts of the jump measure on the quasi-potential quantitatively and on the most probable exit path qualitatively. Results show that the quasi-potential can be well estimated by an approximate analytical expression. Moreover, we discover that although the most probable exit paths are analogous to the Gaussian case for the isotropic noise, the anisotropic noise leads to significant changes of the structure of the exit paths. These findings shed light on the underlying qualitative mechanism and quantitative feature of the exit phenomenon induced by non-Gaussian noise.