We study stochastic partial differential equations (SPDEs) with potentially very rough fractional noise with Hurst parameter $H\in(0,1)$. Close to a change of stability measured with a small parameter $\varepsilon$, we rely on the natural separation of time-scales and establish a simplified description of the essential dynamics. We prove that up to an error term bounded by a power of $\varepsilon$ depending on the Hurst parameter we can approximate the solution of the SPDE in first order by an SDE, the so called amplitude equation, and in second order by a fast infinite dimensional Ornstein-Uhlenbeck process. To this aim we need to establish an explicit averaging result for stochastic integrals driven by rough fractional noise for small Hurst parameters.