Phase-noise (PN) estimation and compensation are crucial in millimeter-wave (mmWave) communication systems to achieve high reliability. The PN estimation, however, suffers from high computational complexity due to its fundamental characteristics, such as spectral spreading and fast-varying fluctuations. In this paper, we propose a new framework for low-complexity PN compensation in orthogonal frequency-division multiplexing systems. The proposed framework also includes a pilot allocation strategy to minimize its overhead. The key ideas are to exploit the coherence bandwidth of mmWave systems and to approximate the actual PN spectrum with its dominant components, resulting in a non-iterative solution by using linear minimum mean squared-error estimation. The proposed method obtains a reduction of more than 2.5x in total complexity, as compared to the existing methods. Furthermore, we derive closed-form expressions for normalized mean squared-errors (NMSEs) as a function of critical system parameters, which help in understanding the NMSE behavior in low and high signal-to-noise ratio regimes. Lastly, we study a trade-off between performance and pilot-overhead to provide insight into an appropriate approximation of the PN spectrum.