Starting with a finite point set $X \subset \mathbf{R}^d$, the peeling process repeatedly removes the set of the vertices of the convex hull of the current set. The number of peeling steps required to completely remove $X$ is called the layer number of $X$, denoted by $L(X)$. In the article, we study the layer number of evenly distributed families of point sets contained in $B^d$, the $d$-dimensional unit ball. These sets consist of points in $B^d$ whose minimal distance is asymptotically as large as possible. We show that for a set $X$ belonging to an evenly distributed family, $L(X) \geq Ω(|X|^{1/d})$ holds, with the bound being asymptotically sharp. On the other hand, building on earlier results, we prove that $L(X)\leq O(|X|^{2/d})$ holds for $d\geq 2$, which improves greatly on the current upper bound of $O(|X|^{(d+1)/2d})$ for $d \geq 3$. Finally, we provide a recursive construction of evenly distributed families whose sets satisfy $L(X) = Θ(|X|^{2/d - 1/(d 2^{d-1})})$, showing that our upper bound is nearly tight.