This paper presents a factor graph formulation and particle-based sum-product algorithm (SPA) for robust sequential localization in multipath-prone environments. The proposed algorithm jointly performs data association, sequential estimation of a mobile agent position, and adapts all relevant model parameters. We derive a novel non-uniform false alarm (FA) model that captures the delay and amplitude statistics of the multipath radio channel. This model enables the algorithm to indirectly exploit position-related information contained in the multipath components (MPCs) for the estimation of the agent position without using any prior information such as floorplan information or training data. Using simulated and real measurements in different channel conditions, we demonstrate that the algorithm can provide high-accuracy position estimates even in fully obstructed line-of-sight (OLOS) situations and show that the performance of our algorithm constantly attains the posterior Cramer-Rao lower bound (P-CRLB), facilitating the additional information contained in the presented FA model. The algorithm is shown to provide robust estimates in both, dense multipath channels as well as channels showing specular, resolved MPCs, significantly outperforming state-of-the-art radio-based localization methods.