In this correspondence, we propose a new receiver design for high-mobility orthogonal frequency division multiplexing (OFDM) downlink transmissions with a large-scale antenna array. The downlink signal experiences the challenging fast time-varying propagation channel. The time-varying nature originates from the multiple carrier frequency offsets (CFOs) due to the transceiver oscillator frequency offset (OFO) and multiple Doppler shifts. Let the received signal first go through a carefully designed beamforming network, which could separate multiple CFOs in the spatial domain with sufficient number of receive antennas. A joint estimation method for the Doppler shifts and the OFO is further developed. Then the conventional single-CFO compensation and channel estimation method can be carried out for each beamforming branch. The proposed receiver design avoids the complicated time-varying channel estimation, which differs a lot from the conventional methods. More importantly, the proposed scheme can be applied to the commonly used time-varying channel models, such as the Jakes' channel model.