The concepts of mean (i.e., average) and covariance of a random variable are fundamental in statistics, and are used to solve real-world problems such as those that arise in robotics, computer vision, and medical imaging. On matrix Lie groups, multiple competing definitions of the mean arise, including the Euclidean, projected, distance-based (i.e., Fréchet and Karcher), group-theoretic, and parametric means. This article provides a comprehensive review of these definitions, investigates their relationships to each other, and determines the conditions under which the group-theoretic means minimize a least-squares type cost function. We also highlight the dependence of these definitions on the choice of inner product on the Lie algebra. The goal of this article is to guide practitioners in selecting an appropriate notion of the mean in applications involving matrix Lie groups.