Millimeter-wave (mmWave) communication is considered as a key enabler of ultra-high data rates in the future cellular and wireless networks. The need for directional communication between base stations (BSs) and users in mmWave systems, that is achieved through beamforming, increases the complexity of the channel estimation. Moreover, in order to provide better coverage, dense deployment of BSs is required which causes frequent handovers and increased association overhead. In this paper, we present an approach that jointly addresses the beamforming and handover problems. Our solution entails an efficient beamforming method with a minimum number of pilots and a learning-based handover method supporting mobile scenarios. We use reinforcement learning algorithm to learn the optimal choices of the backup BSs in different locations of a mobile user. We show that our method provides high rate and reliability in all locations of the user's trajectory with a minimal number of handovers. Simulation results in an outdoor environment based on geometric mmWave channel modeling and real building map data show the superior performance of our proposed solution in achievable instantaneous rate and trajectory rate.