Highly automated driving functions currently often rely on a-priori knowledge from maps for planning and prediction in complex scenarios like cities. This makes map-relative localization an essential skill. In this paper, we address the problem of localization with automotive-grade radars, using a real-time graph-based SLAM approach. The system uses landmarks and odometry information as an abstraction layer. This way, besides radars, all kind of different sensor modalities including cameras and lidars can contribute. A single, semantic landmark map is used and maintained for all sensors. We implemented our approach using C++ and thoroughly tested it on data obtained with our test vehicles, comprising cars and trucks. Test scenarios include inner cities and industrial areas like container terminals. The experiments presented in this paper suggest that the approach is able to provide a precise and stable pose in structured environments, using radar data alone. The fusion of additional sensor information from cameras or lidars further boost performance, providing reliable semantic information needed for automated mapping.