For the large family of ARMA models with variable coefficients (TV-ARMA), either deterministic or stochastic, we provide an explicit and computationally tractable representation based on the general solution of the associated linear difference equation. Analogous representations are established for the fundamental properties of such processes, including the Wold-Cramér decomposition and their covariance structure as well as explicit optimal linear forecasts based on a finite set of past observations. These results are grounded on the principal determinant, that is a banded Hessenbergian representation of a restriction of the Green function involved in the solution of the linear difference equation associated with TV-ARMA models, built up solely of the autoregressive coefficients of the model. The $L_2$ convergence properties of the model are a consequence of the absolute summability of the aforementioned Hessenbergian representation, which is in line with the asymptotic stability and efficiency of such processes. The invertibility of the model is also a consequence of an analogous condition, but now the Green function is built up of the moving average coefficients. The structural asymmetry between constant and deterministically time-varying coefficient models, that is the backward and forward asymptotic efficiency differ in an essential manner, is formally demonstrated. An alternative approach to the Hessenbergian solution representation is described by an equivalent procedure for manipulating time-varying polynomials. The practical significance of the theoretical results in this work is illustrated with an application to U.S. inflation data. The main finding is that inflation persistence increased after 1976, whereas from 1986 onwards the persistence declines and stabilizes to even lower levels than the pre-1976 period.