Precise extraction of sinusoidal vibration parameters is essential for the dynamic calibration of vibration sensors, such as accelerometers. However, several standard methods have not yet been optimized for large background noise. In this work, signal processing methods to extract small vibration signals from noisy data in the case of accelerometer calibration is discussed. The results show that spectral leakage degrades calibration accuracy. Three methods based on the use of a filter, window function, and numerical differentiation are investigated with theoretical calculations, simulations, and experiments. These methods can effectively reduce the contribution of the calibration system noise. The uncertainty of micro vibration calibration in the National Metrology Institute of Japan is reduced by two orders of magnitudes using the proposed methods. The theoretical analyses in this work can lay the foundation for the optimization of signal processing in vibration calibration, and can be applied to other dynamic calibration fields.