For $n\ge 5$, we prove that every $n\times n$ matrix $M=(a_{i,j})$ with entries in $\{-1,1\}$ and absolute discrepancy $|\mathrm{disc}(M)|=|\sum a_{i,j}|\le n$ contains a zero-sum square except for the split matrix (up to symmetries). Here, a square is a $2\times 2$ sub-matrix of $M$ with entries $a_{i,j}, a_{i+s,s}, a_{i,j+s}, a_{i+s,j+s}$ for some $s\ge 1$, and a split matrix is a matrix with all entries above the diagonal equal to $-1$ and all remaining entries equal to $1$. In particular, we show that for $n\ge 5$ every zero-sum $n\times n$ matrix with entries in $\{-1,1\}$ contains a zero-sum square.