Markov Chain Monte Carlo methods are algorithms used to sample probability distributions, commonly used to sample the Boltzmann distribution of physical/chemical models (e.g., protein folding, Ising model, etc.). This allows us to study their properties by sampling the most probable states of those systems. However, the sampling capabilities of these methods are not sufficiently accurate when handling complex configuration spaces. This has resulted in the development of new techniques that improve sampling accuracy, usually at the expense of increasing the computational cost. One of such techniques is Parallel Tempering which improves accuracy by running several replicas which periodically exchange their states. Computationally, this imposes a significant slow-down, which can be counteracted by means of parallelization. These schemes enable MCMC/PT techniques to be run more effectively and allow larger models to be studied. In this work, we present a parallel implementation of Metropolis-Hastings with Parallel Tempering, using OpenMP and CUDA for the parallelization in modern CPUs and GPUs, respectively. The results show a maximum speed-up of 52x using OpenMP with 48 cores, and of 986x speed-up with the CUDA version. Furthermore, the results serve as a basic benchmark to compare a future quantum implementation of the same algorithm.