Differential Evolution (DE) is a highly successful population based global optimisation algorithm, commonly used for solving numerical optimisation problems. However, as the complexity of the objective function increases, the wall-clock run-time of the algorithm suffers as many fitness function evaluations must take place to effectively explore the search space. Due to the inherently parallel nature of the DE algorithm, graphics processing units (GPU) have been used to effectively accelerate both the fitness evaluation and DE algorithm. This work reviews the main architectural choices made in the literature for GPU based DE algorithms and introduces a new GPU based numerical optimisation benchmark to evaluate and compare GPU based DE algorithms.