Switching ARMA models greatly enhance the standard linear models to the extent that different ARMA model is allowed in a different regime, and the regime switching is typically assumed a Markov chain on the finite states of potential regimes. Although statistical issues have been the subject of many recent papers, there is few systematic study of the probabilistic aspects of this new class of nonlinear models. This paper discusses some basic issues concerning this class of models including strict stationarity, influence of initial conditions, and second-order property by studying SVAR models. A number of examples are given to illustrate the theory and the variety of applications. Extensions to other models such as mean-shifting, and inhomogeneous transition probabilities are discussed.