Let $d \ge 2$ be a positive integer. We show that for a class of notions $R$ of rank for order-$d$ tensors, which includes in particular the tensor rank, the slice rank and the partition rank, there exist functions $F_{d,R}$ and $G_{d,R}$ such that if an order-$d$ tensor has $R$-rank at least $G_{d,R}(l)$ then we can restrict its entries to a product of sets $X_1 \times \dots \times X_d$ such that the restriction has $R$-rank at least $l$ and the sets $X_1, \dots, X_d$ each have size at most $F_{d,R}(l)$. Furthermore, our proof methods allow us to show that under a very natural condition we can require the sets $X_1, \dots, X_d$ to be pairwise disjoint.