We present a general approach for studying autoregressive categorical time series models with dependence of infinite order and defined conditional on an exogenous covariate process. To this end, we adapt a coupling approach, developed in the literature for bounding the relaxation speed of a chain with complete connection and from which we derive a perturbation result for non-homogenous versions of such chains. We then study stationarity, ergodicity and dependence properties of some chains with complete connections and exogenous covariates. As a consequence, we obtain a general framework for studying some observation-driven time series models used both in statistics and econometrics but without theoretical support.