We study the dependence of mild solutions to linear stochastic evolution equations on Hilbert space driven by Wiener noise, with drift having linear part of the type $A+\varepsilon G$, on the parameter $\varepsilon$. In particular, we study the limit and the asymptotic expansions in powers of $\varepsilon$ of these solutions, as well as of functionals thereof, as $\varepsilon \to 0$, with good control on the remainder. These convergence and series expansion results are then applied to a parabolic perturbation of the Musiela SPDE of mathematical finance modeling the dynamics of forward rates.