We develop a quasi-likelihood analysis procedure for a general class of multivariate marked point processes. As a by-product of the general method, we establish under stability and ergodicity conditions the local asymptotic normality of the quasi-log likelihood, along with the convergence of moments of quasi-likelihood and quasi-Bayesian estimators. To illustrate the general approach, we then turn our attention to a class of multivariate marked Hawkes processes with generalized exponential kernels, comprising among others the so-called Erlang kernels. We provide explicit conditions on the kernel functions and the mark dynamics under which a certain transformation of the original process is Markovian and $V$-geometrically ergodic. We finally prove that the latter result, which is of interest in its own right, constitutes the key ingredient to show that the generalized exponential Hawkes process falls under the scope of application of the quasi-likelihood analysis.