In this paper, we study the transient phase of the Vertex Reinforced Jump Process (VRJP) in dimension $d\geq 3$. In Sabot, Zeng (2019), the authors introduce a positive martingale and show that the VRJP is recurrent if and only if that martingale converges to $0$. On $\mathbb{Z}^d$, $d\ge 3$, with constant conductances $W$, it can be shown that there is a critical value $0<W_c(\mathbb{Z}^d)<\infty$, such that the martingale converges to $0$ if $W<W_c(\mathbb{Z}^d)$ or to a positive limit if $W>W_c(\mathbb{Z}^d)$. On the other hand, the VRJP martingale can be interpreted as the partition function of a non-directed polymer with a very specific $1$-dependent random potential. In this paper, we focus on the question of the $L^p$ integrability of the VRJP martingale, which is related to the (diffusive) behavior of the VRJP. First, taking inspiration from the work of Junk (2022) for directed polymers in $\mathbb{Z}^{1+d}$, we prove that on the half-space $\mathbb{H}_d$ of $\mathbb{Z}^d$, for all $W>W_c(\mathbb{H}_d)$ there is some $δ>0$ such that the VRJP martingale is in $L^{1+δ}$. Second, we prove that, in dimension $d\geq 4$, the VRJP martingale is in $L^{p}$ for all $p>1$ above the ``slab critical point'' $W_c^{\mathrm{slab}} (\mathbb{Z}^d) = \lim_{m\to\infty} W_c(\mathbb{Z}^{d-1} \times \{-m,\ldots,m\})$. We also propose some related conjectures.