In 1950 Edward Nelson asked the following simple-sounding question: \emph{How many colors are needed to color the Euclidean plane $\mathbb{E}^2$ such that no two points distance $1$ apart are identically colored?} We say that $1$ is a \emph{forbidden} distance. For many years, we only knew that the answer was $4$, $5$, $6$, or $7$. In a recent breakthrough, de Grey \cite{degrey} proved that at least five colors are necessary. In this paper we consider a related problem in which we require \emph{two} forbidden distances, $1$ and $d$. In other words, for a given positive number $d\neq 1$, how many colors are needed to color the plane such that no two points distance $1$ \underline{or} $d$ apart are assigned the same color? We find several values of $d$, for which the answer to the previous question is at least $5$. These results and graphs may be useful in constructing simpler $5$-chromatic unit distance graphs.