In this paper we study model reduction of linear and bilinear quadratic stochastic control problems with parameter uncertainties. Specifically, we consider slow-fast systems with unknown diffusion coefficient and study the convergence of the slow process in the limit of infinite scale separation. The aim of our work is two-fold: Firstly, we want to propose a general framework for averaging and homogenisation of multiscale systems with parametric uncertainties in the drift or in the diffusion coefficient. Secondly, we want to use this framework to quantify the uncertainty in the reduced system by deriving a limit equation that represents a worst-case scenario for any given (possibly path-dependent) quantity of interest. We do so by reformulating the slow-fast system as an optimal control problem in which the unknown parameter plays the role of a control variable that can take values in a closed bounded set. For systems with unknown diffusion coefficient, the underlying stochastic control problem admits an interpretation in terms of a stochastic differential equation driven by a G-Brownian motion. We prove convergence of the slow process with respect to the nonlinear expectation on the probability space induced by the G-Brownian motion. The idea here is to formulate the nonlinear dynamic programming equation of the underlying control problem as a forward-backward stochastic differential equation in the G-Brownian motion framework (in brief: G-FBSDE), for which convergence can be proved by standard means. We illustrate the theoretical findings with two simple numerical examples, exploiting the connection between fully nonlinear dynamic programming equations and second-order BSDE (2BSDE): a linear quadratic Gaussian regulator problem and a bilinear multiplicative triad that is a standard benchmark system in turbulence and climate modelling.