In this paper we propose a lift of vector field $X$ on a Riemannian manifold $M$ to a vector field $\tilde{X}$ on the curved Cameron-Martin space $H\left(M\right)$ named orthogonal lift. The construction of this lift is based on a least square spirit with respect to a metric on $H(M)$ reflecting the damping effect of Ricci curvature. Its stochastic extension gives rise to a non-adapted Cameron-Martin vector field on $W_o(M)$. In particular, if $M=\mathbb{R}^d$ with Euclidean metric, then the damp disappears and the lift reduces to the well-known Malliavin's lift. We establish an integration by parts formula for these first order differential operators.