In 5G, beam training consists of the efficient association of users to beams for a given beamforming codebook used at the base station and the given propagation environment in the cell. We propose a convolutional neural network approach that leverages the position of the base station and geospatial data to predict beam distributions for all user locations simultaneously. Our method generalizes to unseen environments without site-specific training or specialized sensors. The results show that it significantly reduces the number of candidate beams considered, thereby improving the efficiency of beam training.