In this paper, we study systems of $N$ interacting particles described by the classical and relativistic Langevin dynamics with singular forces and multiplicative noises. For the classical model, we prove the ergodicity, obtaining an exponential rate of convergence to the invariant Boltzmann-Gibbs distribution, and the small-mass limit, recovering the $N$-particle interacting overdamped Langevin dynamics. For the relativistic model, we establish the ergodicity, obtaining an algebraic mixing rate of any order to the Maxwell-Jüttner distribution, and the Newtonian limit (that is when the speed of light tends to infinity), approximating a system of underdamped Langevin dynamics. The proofs rely on the construction of Lyapunov functions that account for irregular potentials and multiplicative noises.