Risk measures, which typically evaluate the impact of extreme losses, are highly sensitive to misspecification in the tails. This paper studies a robust optimization approach to combat tail uncertainty by proposing a unifying framework to construct uncertainty sets for a broad class of risk measures, given a specified nominal model. Our framework is based on a parametrization of robust risk measures using two (or multiple) $φ$-divergence functions, which enables us to provide uncertainty sets that are tailored to both the sensitivity of each risk measure to tail losses and the tail behavior of the nominal distribution. In addition, our formulation allows for a tractable computation of robust risk measures, and elicitation of $φ$-divergences that describe a decision maker's risk and ambiguity preferences.