In 2015, Brown and Erey conjectured that every $2$-connected graph $G$ on $n$ vertices with chromatic number $k\geq 4$ has at most $(x-1)_{k-1}\big((x-1)^{n-k+1}+(-1)^{n-k}\big)$ proper $x$-colorings for all $x\geq k$. Engbers, Erey, Fox, and He proved this conjecture for $x=k$. In this paper, we prove Brown and Erey's conjecture under the condition that either the clique number of $G$ is $k$, or the independent number of $G$ is $2$.