The goal of this paper is to clarify when the solutions to stochastic partial differential equations stay close to a given subset of the state space for starting points which are close as well. This includes results for deterministic partial differential equations. As an example, we will consider the situation where the subset is a finite dimensional submanifold with boundary. We also discuss applications to mathematical finance, namely the modeling of the evolution of interest rate curves.