Nonequilibrium statistical models of point vortex systems are constructed using an optimal closure method, and these models are employed to approximate the relaxation toward equilibrium of systems governed by the two-dimensional Euler equations, as well as the quasi-geostrophic equations for either single-layer or two-layer flows. Optimal closure refers to a general method of reduction for Hamiltonian systems, in which macroscopic states are required to belong to a parametric family of distributions on phase space. In the case of point vortex ensembles, the macroscopic variables describe the spatially coarse-grained vorticity. Dynamical closure in terms of those macrostates is obtained by optimizing over paths in the parameter space of the reduced model subject to the constraints imposed by conserved quantities. This optimization minimizes a cost functional that quantifies the rate of information loss due to model reduction, meaning that an optimal path represents a macroscopic evolution that is most compatible with the microscopic dynamics in an information-theoretic sense. A near-equilibrium linearization of this method is used to derive dissipative equations for the low-order spatial moments of ensembles of point vortices in the plane. These severely reduced models describe the late-stage evolution of coherent structures in two-dimensional and geostrophic turbulence. For single-layer dynamics, they approximate the relaxation of initially distorted structures toward axisymmetric equilibrium states. For two-layer dynamics, they predict the rate of energy transfer in baroclinically perturbed structures returning to stable barotropic states. Comparisons against direct numerical simulations of the fully-resolved many-vortex dynamics validate the predictive capacity of these reduced models.