This paper is concerned with portfolio selection for an investor with power utility in multi-asset financial markets in a rough stochastic environment. We investigate Merton's portfolio problem for different multivariate Volterra models, covering the rough Heston model. First we consider a class of multivariate affine Volterra models introduced in [E. Abi Jaber et al., SIAM J. Financial Math., 12, 369-409, (2021)]. Based on the classical Wishart model described in [N. Bäuerle and Li, Z., J. Appl. Probab., 50, 1025-1043 (2013)], we then introduce a new matrix-valued stochastic volatility model, where the volatility is driven by a Volterra-Wishart process. Due to the non-Markovianity of the underlying processes, the classical stochastic control approach cannot be applied in these settings. To overcome this issue, we provide a verification argument using calculus of convolutions and resolvents. The resulting optimal strategy can then be expressed explicitly in terms of the solution of a multivariate Riccati-Volterra equation. We thus extend the results obtained by Han and Wong to the multivariate case, avoiding restrictions on the correlation structure linked to the martingale distortion transformation used in [B. Han and Wong, H. Y., Finance Res. Lett., 39 (2021)]. We also provide existence and uniqueness theorems for the occurring Volterra processes and illustrate our results with a numerical study.