In this paper we describe our entry for the VoiceMOS Challenge 2022 for both the main and out-of-domain (OOD) track of the competition. Our system is based on finetuning pre-trained self-supervised waveform prediction models, while improving its generalisation ability through stochastic weight averaging. Further, we use influence functions to identity possible low-quality data within the training set to further increase our model's performance for the OOD track. Our system ranked 5th and joint 7th for the main track and OOD track, respectively.