We study the representation of stationary Gaussian Markov random fields as factors of i.i.d. processes, with a focus on their approximation by finitely dependent distributions. Our model is a Gaussian field on $\mathbf{Z}^d$ such that the conditional law of the field at any site is Gaussian of mean $\varepsilon$ times the average of its neighbours, and of variance 1. Building on coupling-from-the-past (CFTP) techniques, we prove that for sufficiently small $\varepsilon$, the distribution of the field can be written as an explicit factor of an i.i.d. process. Furthermore, we construct approximations by finitely dependent fields that are close in total variation to the original field, with exponential decay when the allowed range of dependence grows. We first do the proof for a truncated version of this Gaussian model, showing in this case that the associated field admits a finitary coding with exponential tails, providing a new application of high-noise condition of Häggström and Steif \cite{HaggstromSteif} for an uncountable state space. The proof for the original model is more intricate. Our approach extends classical CFTP-based constructions by developing a stratified coupling method tailored to the continuous and unbounded nature of the Gaussian setting.