The peeling process is defined as follows: starting with a finite point set $X \subset \mathbb{R}^d$, we repeatedly remove the set of vertices of the convex hull of the current set of points. The number of peeling steps needed to completely delete the set $X$ is called the layer number of $X$. In this paper, we study the layer number of the $d$-dimensional integer grid $[n]^d$. We prove that for every $d \geq 1$, the layer number of $[n]^d$ is at least $Ω\left(n^\frac{2d}{d+1}\right)$. On the other hand, we show that for every $d\geq 3$, it takes at most $O(n^{d - 9/11})$ steps to fully remove $[n]^d$. Our approach is based on an enhancement of the method used by Har-Peled and Lidický for solving the 2-dimensional case.