This paper proves the consistency property for the regularized maximum likelihood estimators (MLEs) of multivariate Hawkes processes (MHPs). It also develops an alternating minimization type algorithm (AA-iPALM) to compute the MLEs with guaranteed global convergence to the set of stationary points. The performance of this AA-iPALM algorithm on both synthetic and real-world data shows that AA-iPALM consistently improves over iPALM and PALM. Moreover, AA-iPALM is able to identify the causality structure in rental listings on craigslist and herd behavior in the NASDAQ ITCH dataset.