Matrix Lie groups are an important class of manifolds commonly used in control and robotics, and optimizing control policies on these manifolds is a fundamental problem. In this work, we propose a novel computationally efficient approach for trajectory optimization on matrix Lie groups using an augmented Lagrangian-based constrained discrete Differential Dynamic Programming (DDP). The method involves lifting the optimization problem to the Lie algebra during the backward pass and retracting back to the manifold during the forward pass. Unlike previous approaches that addressed constraint handling only for specific classes of matrix Lie groups, the proposed method provides a general solution for nonlinear constraint handling across generic matrix Lie groups. We evaluate the effectiveness of the proposed DDP method in handling constraints within a mechanical system characterized by rigid body dynamics in SE(3), assessing its computational efficiency compared to existing direct optimization solvers. Additionally, the method demonstrates robustness under external disturbances when applied as a Lie-algebraic feedback control policy on SE(3), and in optimizing a quadrotor's trajectory in a challenging realistic scenario. Experiments show that the proposed approach effectively manages general constraints defined on configuration, velocity, and inputs during optimization, while also maintaining stability under external disturbances when executing the resultant control policy in closed-loop.