In this paper, we propose a novel variable selection approach in the framework of sparse high-dimensional GLARMA models. It consists in combining the estimation of the autoregressive moving average (ARMA) coefficients of these models with regularized methods designed for Generalized Linear Models (GLM). The properties of our approach are investigated both from a theoretical and a numerical point of view. More precisely, we establish in a specific case the consistency of the ARMA part coefficient estimators. We explain how to implement our approach and we show that it is very attractive since it benefits from a low computational load. We also assess the performance of our methodology using synthetic data and compare it with alternative approaches. Our numerical experiments show that combining the estimation of the ARMA part coefficients with regularized methods designed for GLM dramatically improves the variable selection performance.