The complement $\overline{x}$ of a binary word $x$ is obtained by changing each $0$ in $x$ to $1$ and vice versa. We study infinite binary words $\bf w$ that avoid sufficiently large complementary factors; that is, if $x$ is a factor of $\bf w$ then $\overline{x}$ is not a factor of $\bf w$. In particular, we classify such words according to their critical exponents.