We consider the system-wide utility maximization problem in the downlink of a cell-free massive multiple-input multiple-output (MIMO) system whereby a very large number of access points (APs) simultaneously serve a group of users. Specifically, four fundamental problems with increasing order of user fairness are of interest: (i) to maximize the average spectral efficiency (SE), (ii) to maximize the proportional fairness, (iii) to maximize the harmonic-rate of all users, and lastly (iv) to maximize the minimum SE of all users, subject to a sum power constraint at each AP. As the considered problems are non-convex, existing solutions normally rely on successive convex approximation to find a sub-optimal solution. More specifically, these known methods use off-the-shelf convex solvers, which basically implement an interior-point algorithm, to solve the derived convex problems. The main issue of such methods is that their complexity does not scale favorably with the problem size, limiting previous studies to cell-free massive MIMO of moderate scales. Thus the potential of cell-free massive MIMO has not been fully understood. To address this issue, we propose a unified framework based on an accelerated projected gradient method to solve the considered problems. Particularly, the proposed solution is found in closed-form expressions and only requires the first order oracle of the objective, rather than the Hessian matrix as in known solutions, and thus is much more memory efficient. Numerical results demonstrate that our proposed solution achieves the same utility performance but with far less run-time, compared to other second-order methods. Simulation results for large-scale cell-free massive MIMO show that the four utility functions can deliver nearly uniformed services to all users. In other words, user fairness is not a great concern in large-scale cell-free massive MIMO.