This paper gives the first quantitative bounds for the inverse theorem for the Gowers $U^4$-norm over $\mathbb{F}_p^n$ when $p=2,3$. We build upon earlier work of Gowers and Milićević who solved the corresponding problem for $p\geq 5$. Our proof has two main steps: symmetrization and integration of low-characteristic trilinear forms. We are able to solve the integration problem for all $k$-linear forms, but the symmetrization problem we are only able to solve for trilinear forms. We pose several open problems about symmetrization of low-characteristic $k$-linear forms whose resolution, combined with recent work of Gowers and Milićević, would give quantitative bounds for the inverse theorem for the Gowers $U^{k+1}$-norm over $\mathbb{F}_p^n$ for all $k,p$.