In this article, we study a robust estimation method for a general class of integer-valued time series models. The conditional distribution of the process belongs to a broad class of distribution and unlike classical autoregressive framework, the conditional mean of the process also depends on some multivariate exogenous covariate. We derive a robust inference procedure based on the minimum density power divergence. Under certain regularity conditions, we establish that the proposed estimator is consistent and asymptotically normal. Simulation experiments are conducted to illustrate the empirical performances of the estimator. An application to the number of transactions per minute for the stock Ericsson B is also provided.