The restricted $h$-connectivity of a graph $G$, denoted by $κ^h(G)$, is defined as the minimum cardinality of a set of vertices $F$ in $G$, if exists, whose removal disconnects $G$ and the minimum degree of each component of $G-F$ is at least $h$. In this paper, we study the restricted $h$-connectivity of the balanced hypercube $BH_n$ and determine that $κ^1(BH_n)=κ^2(BH_n)=4n-4$ for $n\geq2$. We also obtain a sharp upper bound of $κ^3(BH_n)$ and $κ^4(BH_n)$ of $n$-dimension balanced hypercube for $n\geq3$ ($n\neq4$). In particular, we show that $κ^3(BH_3)=κ^4(BH_3)=12$.