We initiate the study of the binary and Boolean rank of $0,1$ matrices that have a small rank over the reals. The relationship between these three rank functions is an important open question, and here we prove that when the real rank $d$ is a small constant, the gap between the real and the binary and Boolean rank is a small constant. We give tight upper and lower bounds on the Boolean and binary rank of matrices with real rank $1 \leq d \leq 4$, as well as determine the size of the largest isolation set in each case. Furthermore, we prove that for $d = 3,4$, the circulant matrix defined by a row with $d-1$ consecutive ones followed by $d-1$ zeros, is the only matrix of size $(2d-2)\times (2d-2)$ with real rank $d$ and Boolean and binary rank and isolation set of size $2d-2$, and this matrix achieves the maximal gap possible between the real and the binary and Boolean rank for these values of $d$. Our results can also be interpreted in other equivalent terms, such as finding the minimal number of bicliques needed to partition or cover the edges of a bipartite graph whose reduced adjacency matrix has real rank $1 \leq d \leq 4$. We use a combination of combinatorial and algebraic techniques combined with the assistance of a computer program.