This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. Numerous renowned algorithms for tackling the compressed sensing problem employ an alternating strategy, which typically involves data matching in one module and denoising in another. We present a novel approach, the Alternating Subspace Method (ASM), which integrates the principles of the greedy methods (e.g., the orthogonal matching pursuit type methods) and the splitting methods (e.g., the approximate message passing type methods). Crucially, ASM enhances the splitting method by achieving fidelity in a subspace-restricted fashion. \textcolor{black}{We reveal that such a restriction strategy guarantees global convergence via proximal residual control and establish its local geometric convergence on the LASSO problem.} Numerical experiments on the LASSO, channel estimation, and dynamic compressed sensing problems demonstrate its high convergence rate and its capacity to incorporate different prior distributions. Overall, the proposed method is promising in terms of efficiency, accuracy, and flexibility, and has the potential to be competitive in different sparse recovery applications.