Many standard estimators such as several maximum likelihood estimators or the empirical estimator for any law-invariant convex risk measure are not (qualitatively) robust in the classical sense. However, these estimators may nevertheless satisfy a weak (Krätschmer/Schied/Zähle 2012, 2014) or a local (Zähle 2016) robustness property on relevant sets of distributions. One aim of our paper is to identify sets of local robustness, and to explain the benefit of the knowledge of such sets. For instance, we will be able to demonstrate that many maximum likelihood estimators are robust on their natural parametric domains. A second aim consists in extending the general theory of robust estimation to our local framework. In particular we provide a corresponding Hampel-type theorem linking local robustness of a plug-in estimator with a certain continuity condition.