We propose a stochastic process driven by memory effect with novel distributions including both exponential and leptokurtic heavy-tailed distributions. A class of distribution is analytically derived from the continuum limit of the discrete binary process with the renormalized auto-correlation and the closed form moment generating function is obtained, thus the cumulants are calculated and shown to be convergent. The other class of distributions are numerically investigated. The concoction of the two stochastic processes of the different signs of memory under regime switching mechanism does incarnate power-law decay behavior, which strongly implies that memory is the alternative origin of heavy-tail.