When estimating the risk of a P&L from historical data or Monte Carlo simulation, the robustness of the estimate is important. We argue here that Hampel's classical notion of qualitative robustness is not suitable for risk measurement and we propose and analyze a refined notion of robustness that applies to tail-dependent law-invariant convex risk measures on Orlicz space. This concept of robustness captures the tradeoff between robustness and sensitivity and can be quantified by an index of qualitative robustness. By means of this index, we can compare various risk measures, such as distortion risk measures, in regard to their degree of robustness. Our analysis also yields results that are of independent interest such as continuity properties and consistency of estimators for risk measures, or a Skorohod representation theorem for ψ-weak convergence.