The focus of this paper is on beamforming in a millimeter-wave (mmW) multi-input multi-output (MIMO) setup that has gained increasing traction in meeting the high data-rate requirements of next-generation wireless systems. For a given MIMO channel matrix, the optimality of beamforming with the dominant right-singular vector (RSV) at the transmit end and with the matched filter to the RSV at the receive end has been well-understood. When the channel matrix can be accurately captured by a physical (geometric) scattering model across multiple clusters/paths as is the case in mmW MIMO systems, we provide a physical interpretation for this optimal structure: beam steering across the different paths with appropriate power allocation and phase compensation. While such an explicit physical interpretation has not been provided hitherto, practical implementation of such a structure in a mmW system is fraught with considerable difficulties (complexity as well as cost) as it requires the use of per-antenna gain and phase control. This paper characterizes the loss in received SNR with an alternate low-complexity beamforming solution that needs only per-antenna phase control and corresponds to steering the beam to the dominant path at the transmit and receive ends. While the loss in received SNR can be arbitrarily large (theoretically), this loss is minimal in a large fraction of the channel realizations reinforcing the utility of directional beamforming as a good candidate solution for mmW MIMO systems.