For every $d\geq 1$, we consider the $d$-dimensional Hermitian fractional Brownian motion (HfBm), that is the process with values in the space of $(d\times d)$-Hermitian matrices and with upper-diagonal entries given by complex fractional Brownian motions of Hurst index $H\in (0,1)$. We follow the approach of [A. Deya and R. Schott: On the rough paths approach to non-commutative stochastic calculus, JFA (2013)] to define a natural integral with respect to the HfBm when $H>\frac13$, and identify this interpretation with the rough integral with respect to the $d^2$ entries of the matrix. Using this correspondence, we establish a convenient It{ô}--Stratonovich formula for the Hermitian Brownian motion. Finally, we show that at least when $H\geq \frac12$, and as the size $d$ of the matrix tends to infinity, the integral with respect to the HfBm converges (in the tracial sense) to the integral with respect to the so-called non-commutative fractional Brownian motion.