Because of the relatively rigid coupling between the upper dentition and the skull, instrumented mouthguards have been shown to be a viable way of measuring head impact kinematics for assisting in understanding the underlying biomechanics of concussions. This has led various companies and institutions to further develop instrumented mouthguards. However, their use as a research tool for understanding concussive impacts makes quantification of their accuracy critical, especially given the conflicting results from various recent studies. Here we present a study that uses a pneumatic impactor to deliver impacts characteristic to football to a Hybrid III headform, in order to validate and compare five of the most commonly used instrumented mouthguards. We found that all tested mouthguards gave accurate measurements for the peak angular acceleration (mean relative error, MRE < 13%), the peak angular velocity (MRE < 8%), brain injury criteria values (MRE < 13%) and brain deformation (described as maximum principal strain and fiber strain, calculated by a convolutional neural network based brain model, MRE < 9%). Finally, we found that the accuracy of the measurement varies with the impact locations yet is not sensitive to the impact velocity for the most part.