It is well-known that combining life annuities and death benefits introduce opposite effects in payments with respect to the mortality risk on the lifetime of the insured. In a general multi-state framework with multiple product types, such joint effects are less trivial. In this paper, we consider a multivariate payment process in multi-state life insurance, where the components are defined in terms of the same Markovian state process. The multivariate present value of future payments is introduced, and we derive differential equations and product integral representations of its conditional moments and moment generating function. Special attention is given to pair-wise covariances between two present values, where results closely connected to Hattendorff type of results for the variance are derived. The results are illustrated in a numerical example in a disability model.