We consider the utility maximization problem under convex constraints with regard to theoretical results which allow the formulation of algorithmic solvers which make use of deep learning techniques. In particular for the case of random coefficients, we prove a stochastic maximum principle (SMP), which also holds for utility functions $U$ with $\mathrm{id}_{\mathbb{R}^{+}} \cdot U'$ being not necessarily nonincreasing, like the power utility functions, thereby generalizing the SMP proved by Li and Zheng (2018). We use this SMP together with the strong duality property for defining a new algorithm, which we call deep primal SMP algorithm. Numerical examples illustrate the effectiveness of the proposed algorithm - in particular for higher-dimensional problems and problems with random coefficients, which are either path dependent or satisfy their own SDEs. Moreover, our numerical experiments for constrained problems show that the novel deep primal SMP algorithm overcomes the deep SMP algorithm's (see Davey and Zheng (2021)) weakness of erroneously producing the value of the corresponding unconstrained problem. Furthermore, in contrast to the deep controlled 2BSDE algorithm from Davey and Zheng (2021), this algorithm is also applicable to problems with path dependent coefficients. As the deep primal SMP algorithm even yields the most accurate results in many of our studied problems, we can highly recommend its usage. Moreover, we propose a learning procedure based on epochs which improved the results of our algorithm even further. Implementing a semi-recurrent network architecture for the control process turned out to be also a valuable advancement.