In this paper, quantitative bounds in high-frequency central limit theorems are derived for Poisson based $U$-statistics of arbitrary degree built by means of wavelet coefficients over compact Riemannian manifolds. The wavelets considered here are the so-called needlets, characterized by strong concentration properties and by an exact reconstruction formula. Furthermore, we consider Poisson point processes over the manifold such that the density function associated to its control measure lives in a Besov space. The main findings of this paper include new rates of convergence that depend strongly on the degree of regularity of the control measure of the underlying Poisson point process, providing a refined understanding of the connection between regularity and speed of convergence in this framework.