Let $G$ be a simple Lie group. Consider a nilpotent element $e\in \mathfrak{g}$. Let $Z_G(e)$ be the centralizer of $e$ in $G$, and let $A_e:= Z_G(e)/Z_G(e)^{o}$ be its component group. Write $\text{Irr}(\mathcal{B}_e)$ for the set of irreducible components of the Springer fiber $\mathcal{B}_e$. We have an action of $A_e$ on $\text{Irr}(\mathcal{B}_e)$. When $\mathfrak{g}$ is exceptional, we give an explicit description of $\text{Irr}(\mathcal{B}_e)$ as an $A_e$-set. For $\mathfrak{g}$ of classical type, we describe the stabilizers for the $A_e$-action. With this description, we prove a conjecture of Lusztig and Sommers. These results suggest relations (first proposed by Lusztig) between Springer fibers and cells in Weyl groups.