Millimeter wave (mmWave) multiple-input multiple-output (MIMO) systems operate over wide bandwidths and frequency-selective channels, making orthogonal frequency-division multiplexing (OFDM) a natural transmission scheme. In such systems, fully digital precoding is often impractical because the large antenna arrays require high hardware cost and power consumption, so hybrid precoding that combines digital and radio frequency (RF) processing is an attractive alternative. However, OFDM introduces high signal peaks that may cause clipping and generate out-of-band (OOB) emissions, while practical, nonideal phase shifters (PSs) at the RF precoder and user combiner suffer from phase errors. We study the problem of robust digital-RF precoding optimization for the downlink sum-rate maximization in multi-user (MU) MIMO-OFDM systems under maximum transmit power, clipping, and OOB emission mask constraints. The formulated maximization problem is nonconvex and difficult to solve. We propose a weighted minimum mean squared error (WMMSE) based block coordinate descent (BCD) method to iteratively optimize digital-RF precoders at the transmitter and digital-RF combiners at the users. Low-cost and scalable optimization approaches are proposed to efficiently solve the BCD subproblems. Extensive simulation results are conducted to demonstrate the efficiency of the proposed approaches and exhibit their superiority relative to well-known benchmarks.