In this paper, we propose a new hybrid analog and digital combining architecture for millimeter wave (mmWave) multi-user multiple-input multiple-output (MU-MIMO) systems. The proposed structure employs antenna subset selection per radio frequency (RF) chain based on active/inactive switches and uses constant phase shifters (CPS) to control the phases of signals in the RF circuit. In this scheme, for each RF chain, a subset of receive antennas that contribute more to the desired signal power than the interference power is chosen for signal combining in the analog domain, whereas other receive antennas are excluded from signal combining, thereby enhancing sum-rates. Simultaneously, the proposed structure reduces power consumption in the RF circuit by exclusively activating switches that correspond to the antennas selected for each RF chain. We also develop three low-complexity algorithms for per-RF chain antenna subset selection. Finally, through numerical simulation, we show that the proposed structure provides higher spectral efficiency and higher energy efficiency than conventional hybrid analog and digital combining schemes for mmWave MU-MIMO systems.