This paper aims to investigate the non-Markovian dynamics. The governing equations are derived for the probability density functions (PDFs) of non-Markovian stochastic responses to Langevin equation excited by combined fractional Gaussian noise (FGN) and Gaussian white noise (GWN). The main difficulty here is that the Langevin equation excited by FGN cannot be augmented by a filter excited by GWN, leading to the inapplicability of Itô stochastic calculus theory. Thus, in the present work, based on the fractional Wick Itô Skorohod integral and rough path theory, a new non-Markovian probability density evolution method is established to derive theoretically the memory-dependent probability density evolution equation (PDEEs) for the PDFs of non-Markovian stochastic responses to Langevin equation excited by combined FGN and GWN, which is a breakthrough to stochastic dynamics. Then, we extend an efficient algorithm, the local discontinuous Galerkin method, to numerically solve the memory-dependent PDEEs. Remarkably, this proposed method attains a higher accuracy compared to the prevalent methods such as finite difference, path integral (PI) and Monte Carlo methods, and boasts a broader applicability than the PI method, which fails to solve the memory-dependent PDEEs. Finally, several numerical examples are illustrated to verify the proposed scheme.