In this paper, we prove a general result concerning finite-range, attractive interacting particle systems on $\{-1, 1\}^{\mathbb{Z}^d}$. If the particle system has a unique stationary measure and, in a precise sense, relaxes to this stationary measure at an exponential rate then any small perturbation of the dynamics also has a unique stationary measure to which it relaxes at an exponential rate. To augment this result, we study the particular case of Glauber dynamics for the Ising model. We show that for any non-zero external field the dynamics converges to its unique invariant measure at an exponential rate. Previously, this was only known for $β<β_c$ and $β$ sufficiently large. As a consequence, Glauber dynamics is stable to small, non-equilibrium perturbations in the entire uniqueness phase.