In first-passage percolation, one assigns i.i.d. nonnegative weights $(t_e)$ to the edges of $\mathbb{Z}^d$ and studies the induced distance (passage time) $T(x,y)$ between vertices $x$ and $y$. It is known that for $d=2$, the fluctuations of $T(x,y)$ are at least order $\sqrt{\log |x-y|}$ under mild assumptions on $t_e$. We study the question of fluctuation lower bounds for $T_n$, the minimal passage time between two opposite sides of an $n$ by $n$ square. The main result is that, under a curvature assumption, this quantity has fluctuations at least of order $n^{1/8-ε}$ for any $ε>0$ when the $t_e$ are exponentially distributed. As previous arguments to bound the fluctuations of $T(x,y)$ only give a constant lower bound for those of $T_n$ (even assuming curvature), a different argument, representing $T_n$ as a minimum of cylinder passage times, and deriving more detailed information about the distribution of cylinder times using the Markov property, is developed. As a corollary, we obtain the first polynomial lower bounds on higher central moments of the discrete torus passage time, under the same curvature assumption.