Using Jack polynomials, Goulden and Jackson have introduced a one parameter deformation $τ_b$ of the generating series of bipartite maps, which generalizes the partition function of $β$-ensembles of random matrices. The Matching-Jack conjecture suggests that the coefficients $c^λ_{μ,ν}$ of the function $τ_b$ in the power-sum basis are non-negative integer polynomials in the deformation parameter $b$. Dołęga and Féray have proved in 2016 the "polynomiality" part in the Matching-Jack conjecture, namely that coefficients $c^λ_{μ,ν}$ are in $\mathbb{Q}[b]$. In this paper, we prove the "integrality" part, i.e that the coefficients $c^λ_{μ,ν}$ are in $\mathbb{Z}[b]$. The proof is based on a recent work of the author that deduces the Matching-Jack conjecture for marginal sums from an analog result for the $b$-conjecture, established in 2020 by Chapuy and Dołęga. A key step in the proof involves a new connection with the graded Farahat-Higman algebra.