We study first-passage percolation on $\mathbb Z ^2$ with independent and identically distributed weights, whose common distribution is uniform on $\{a,b\}$ with $0<a<b<\infty $. Following Ahlberg and De la Riva, we consider the passage time $τ(n,k)$ of the minimal left-right crossing of the square $[0,n]^2$, whose vertical fluctuations are bounded by $k$. We prove that when $k\le n^{1/2-ε}$, the event that $τ(n,k)$ is larger than its median is noise sensitive. This improves the main result of Ahlberg and De la Riva which holds when $k\le n^{1/22-ε}$. Under the additional assumption that the limit shape is not a polygon with a small number of sides, we extend the result to all $k\le n^{1-ε}$. This extension follows unconditionally when $a$ and $b$ are sufficiently close. Under a stronger curvature assumption, we extend the result to all $k\le n$. This in particular captures the noise sensitivity of the event that the minimal left-right crossing $T_n=τ(n,n)$ is larger than its median. Finally, under the curvature assumption, our methods give a lower bound of $n^{1/4-ε}$ for the variance of the passage time $T_n$ of the minimal left-right crossing of the square. We prove the last bound also for absolutely continuous weight distributions, generalizing a result of Damron--Houdré--Özdemir, which holds only for the exponential distribution. Our approach differs from the previous works mentioned above; the key idea is to establish a small ball probability estimate in the tail by perturbing the weights for tail events using a Mermin--Wagner type estimate.