The Langevin Dynamics (LD), which aims to sample from a probability distribution using its score function, has been widely used for analyzing and developing score-based generative modeling algorithms. While the convergence behavior of LD in sampling from a uni-modal distribution has been extensively studied in the literature, the analysis of LD under a mixture distribution with distinct modes remains underexplored in the literature. In this work, we analyze LD in sampling from a mixture distribution and theoretically study its convergence properties. Our theoretical results indicate that for general mixture distributions of sub-Gaussian components, LD could fail in finding all the components within a sub-exponential number of steps in the data dimension. Following our result on the complexity of LD in sampling from high-dimensional variables, we propose Chained Langevin Dynamics (Chained-LD), which divides the data vector into patches of smaller sizes and generates every patch sequentially conditioned on the previous patches. Our theoretical analysis of Chained-LD indicates its faster convergence speed to the components of a mixture distribution. We present the results of several numerical experiments on synthetic and real image datasets, validating our theoretical results on the iteration complexities of sample generation from mixture distributions using the vanilla and chained LD algorithms.