We express correlators of the Jacobi $β$ ensemble in terms of (a special case of) $b$-Hurwitz numbers, a deformation of Hurwitz numbers recently introduced by Chapuy and Dolega. The proof relies on Kadell's generalization of the Selberg integral. The Laguerre limit is also considered. All the relevant $b$-Hurwitz numbers are interpreted (following Bonzom, Chapuy, and Dolega) in terms of colored monotone Hurwitz maps.