Sparse binary matrices are of great interest in the field of sparse recovery, nonnegative compressed sensing, statistics in networks, and theoretical computer science. This class of matrices makes it possible to perform signal recovery with lower storage costs and faster decoding algorithms. In particular, Bernoulli$(p)$ matrices formed by independent identically distributed (i.i.d.) Bernoulli$(p)$ random variables are of practical relevance in the context of noise-blind recovery in nonnegative compressed sensing. In this work, we investigate the robust nullspace property of Bernoulli$(p)$ matrices. Previous results in the literature establish that such matrices can accurately recover $n$-dimensional $s$-sparse vectors with $m=O\left(\frac{s}{c(p)}\log\frac{en}{s}\right)$ measurements, where $c(p) \le p$ is a constant dependent only on the parameter $p$. These results suggest that in the sparse regime, as $p$ approaches zero, the (sparse) Bernoulli$(p)$ matrix requires significantly more measurements than the minimal necessary, as achieved by standard isotropic subgaussian designs. However, we show that this is not the case. Our main result characterizes, for a wide range of sparsity levels $s$, the smallest $p$ for which sparse recovery can be achieved with the minimal number of measurements. We also provide matching lower bounds to establish the optimality of our results and explore connections with the theory of invertibility of discrete random matrices and integer compressed sensing.