Compressive Robust Principal Component Analysis (CRPCA) naturally arises in various applications as a means to recover a low-rank matrix low-rank matrix $\boldsymbol{L}$ and a sparse matrix $\boldsymbol{S}$ from compressive measurements. In this paper, we approach the problem from a Bayesian inference perspective. We establish a probabilistic model for the problem and develop an improved turbo message passing (ITMP) algorithm based on the sum-product rule and the appropriate approximations. Additionally, we establish a state evolution framework to characterize the asymptotic behavior of the ITMP algorithm in the large-system limit. By analyzing the established state evolution, we further propose sufficient conditions for the global convergence of our algorithm. Our numerical results validate the theoretical results, demonstrating that the proposed asymptotic framework accurately characterize the dynamical behavior of the ITMP algorithm, and the phase transition curve specified by the sufficient condition agrees well with numerical simulations.