This paper concerns a class of DC composite optimization problems which, as an extension of convex composite optimization problems and DC programs with nonsmooth components, often arises in robust factorization models of low-rank matrix recovery. For this class of nonconvex and nonsmooth problems, we propose an inexact linearized proximal algorithm (iLPA) by computing at each step an inexact minimizer of a strongly convex majorization constructed with a partial linearization of their objective functions at the current iterate. We establish the full convergence of the generated iterate sequence under the Kurdyka-Łöjasiewicz (KL) property of a potential function, and employ the composite structure to provide a verifiable condition for the potential function to satisfy the KL property of exponent $1/2$ at the limit point, so for the iterate sequence to have a local R-linear convergence rate. This condition is weaker than the one provided in \cite[Theorem 3.2]{LiPong18} for identifying the KL property of exponent $p\in[0,1)$ for a general composite function. The proposed iLPA is applied to a robust factorization model for matrix completion with outliers and non-uniform sampling, and numerical comparisons with the Polyak subgradient method and a proximal alternating minimization (PAM) method validate its efficiency.