During contagion phenomena, individuals perceiving a risk of infection commonly adapt their behavior and reduce their exposure. The effects of such adaptive mechanisms have been studied for processes in which pairwise interactions drive contagion. However, contagion and the perception of infection risk can also involve ("higher-order") group interactions, leading potentially to new phenomenology. How adaptive behavior resulting from risk perception affects higher-order processes remains an open question. Here, we consider the impact of several risk-based adaptive behaviors on pairwise and higher-order contagion processes, using numerical simulations and an analytical mean-field approach. For pairwise contagion, adaptive mechanisms based on local (pairwise or group-based) risk perception impact only the endemic state, without affecting the epidemic phase transition. For higher-order contagion processes, instead, the adaptivity defuses the impact of non-linear group interactions: this reduces or even completely suppresses the parameter range in which bistability is possible, effectively transforming a higher-order contagion process into a pairwise one.