In this letter, we consider optimal hybrid beamforming design to minimize the transmission power under individual signal-to-interference-plus-noise ratio (SINR) constraints in a multiuser massive multiple-input-multiple-output (MIMO) system. This results in a challenging non-convex optimization problem. We consider two cases. In the case where the number of users is smaller than or equal to that of radio frequency (RF) chains, we propose a low-complexity method to obtain a globally optimal solution and show that it achieves the same transmission power as an optimal fully-digital beamformer. In the case where the number of users is larger than that of RF chains, we propose a low-complexity globally convergent alternating algorithm to obtain a stationary point.