We consider first-passage percolation on $\mathbb{Z}^2$ with i.i.d. weights, whose distribution function satisfies $F(0) = p_c = 1/2$. This is sometimes known as the "critical case" because large clusters of zero-weight edges force passage times to grow at most logarithmically, giving zero time constant. Denote $T(\mathbf{0}, \partial B(n))$ as the passage time from the origin to the boundary of the box $[-n,n] \times [-n,n]$. We characterize the limit behavior of $T(\mathbf{0}, \partial B(n))$ by conditions on the distribution function $F$. We also give exact conditions under which $T(\mathbf{0}, \partial B(n))$ will have uniformly bounded mean or variance. These results answer several questions of Kesten and Zhang from the '90s and, in particular, disprove a conjecture of Zhang from '99. In the case when both the mean and the variance go to infinity as $n \to \infty$, we prove a CLT under a minimal moment assumption. The main tool involves a new relation between first-passage percolation and invasion percolation: up to a constant factor, the passage time in critical first-passage percolation has the same first-order behavior as the passage time of an optimal path constrained to lie in an embedded invasion cluster.