Existing deep neural network (DNN) based wireless localization approaches typically do not capture uncertainty inherent in their estimates. In this work, we propose and evaluate variational and scalable DNN approaches to measure the uncertainty as a result of changing propagation conditions and the finite number of training samples. Furthermore, we show that data uncertainty is sufficient to capture the uncertainty due to non-line-of-sight (NLOS) and, model uncertainty improves the overall reliability. To assess the robustness due to channel conditions and out-of-set regions, we evaluate the methods on challenging massive multiple-input multiple-output (MIMO) scenarios.