In this paper we present a general framework in which one can rigorously study the effect of spatio-temporal noise on traveling waves, stationary patterns and oscillations that are invariant under the action of a finite-dimensional set of continuous isometries (such as translation or rotation). This formalism can accommodate patterns, waves and oscillations in reaction-diffusion systems and neural field equations. To do this, we define the phase by precisely projecting the infinite-dimensional system onto the manifold of isometries. Two differing types of stochastic phase dynamics are defined: (i) a variational phase, obtained by insisting that the difference between the projection and the original solution is orthogonal to the non-decaying eigenmodes, and (ii) an isochronal phase, defined as the limiting point on manifold obtained by taking $t\to\infty$ in the absence of noise. We outline precise stochastic differential equations for both types of phase. The variational phase SDE is then used to show that the probability of the system leaving the attracting basin of the manifold after an exponentially long period of time (in $ε^{-2}$, the magnitude of the noise) is exponentially unlikely. In the case that the manifold is periodic (such as for spiral waves, spatially-distributed oscillations, or neural-field phenomena on a compact domain), the isochronal phase SDE is used to determine asymptotic limits for the average occupation times of the phase as it wanders in the basin of attraction of the manifold over very long times. In particular, we find that frequently the correlation structure of the noise biases the wandering in a particular direction, such that the noise induces a slow oscillation that would not be present in the absence of noise.