This work investigates the use of machine learning applied to the beam tracking problem in 5G networks and beyond. The goal is to decrease the overhead associated to MIMO millimeter wave beamforming. In comparison to beam selection (also called initial beam acquisition), ML-based beam tracking is less investigated in the literature due to factors such as the lack of comprehensive datasets. One of the contributions of this work is a new public multimodal dataset, which includes images, LIDAR information and GNSS positioning, enabling the evaluation of new data fusion algorithms applied to wireless communications. The work also contributes with an evaluation of the performance of beam tracking algorithms, and associated methodology. When considering as inputs the LIDAR data, the coordinates and the information from previously selected beams, the proposed deep neural network based on ResNet and using LSTM layers, significantly outperformed the other beam tracking models.