We introduce a closed-form expansion for the transition density of elliptic and hypo-elliptic multivariate Stochastic Differential Equations (SDEs), over a period $Δ\in (0,1)$, in terms of powers of $Δ^{j/2}$, $j\ge 0$. Our methodology provides approximations of the transition density, easily evaluated via any software that performs symbolic calculations. A major part of the paper is devoted to an analytical control of the remainder in our expansion for fixed $Δ\in(0,1)$. The obtained error bounds validate theoretically the methodology, by characterising the size of the distance from the true value. It is the first time that such a closed-form expansion becomes available for the important class of hypo-elliptic SDEs, to the best of our knowledge. For elliptic SDEs, closed-form expansions are available, with some works identifying the size of the error for fixed $Δ$, as per our contribution. Our methodology allows for a uniform treatment of elliptic and hypo-elliptic SDEs, when earlier works are intrinsically restricted to an elliptic setting. We show numerical applications highlighting the effectiveness of our method, by carrying out parameter inference for hypo-elliptic SDEs that do not satisfy stated conditions. The latter are sufficient for controlling the remainder terms, but the closed-form expansion itself is applicable in general settings.