A procedure for model reduction of stochastic ordinary differential equations with additive noise was recently introduced in [Colangeli-Duong-Muntean, Journal of Physics A: Mathematical and Theoretical, 2022], based on the Invariant Manifold method and on the Fluctuation-Dissipation relation. A general question thus arises as to whether one can rigorously quantify the error entailed by the use of the reduced dynamics in place of the original one. In this work we provide explicit formulae and estimates of the error in terms of the Wasserstein distance, both in the presence or in the absence of a sharp time-scale separation between the variables to be retained or eliminated from the description, as well as in the long-time behaviour. Keywords: Model reduction, Wasserstein distance, error estimates, coupled Brownian oscillators, invariant manifold, Fluctuation-Dissipation relation.