The Glauber dynamics for the classical $2$-spin Curie-Weiss model on $N$ nodes with inverse temperature $β$ and zero external field is known to mix in time $Θ(N\log N)$ for $β< \frac{1}{2}$, in time $Θ(N^{3/2})$ at $β= \frac{1}{2}$, and in time $\exp(Ω(N))$ for $β>\frac{1}{2}$. In this paper, we consider the $p$-spin generalization of the Curie-Weiss model with an external field $h$, and identify three disjoint regions almost exhausting the parameter space, with the corresponding Glauber dynamics exhibiting three different orders of mixing times in these regions. The construction of these disjoint regions depends on the number of local maximizers of a certain function $H_{β,h,p}$, and the behavior of the second derivative of $H_{β,h,p}$ at such a local maximizer. Specifically, we show that if $H_{β,h,p}$ has a unique local maximizer $m_*$ with $H_{β,h,p}''(m_*) < 0$ and no other stationary point, then the Glauber dynamics mixes in time $Θ(N\log N)$, and if $H_{β,h,p}$ has multiple local maximizers, then the mixing time is $\exp(Ω(N))$. Finally, if $H_{β,h,p}$ has a unique local maximizer $m_*$ with $H_{β,h,p}''(m_*) = 0$, then the mixing time is $Θ(N^{3/2})$. We provide an explicit description of the geometry of these three different phases in the parameter space, and observe that the only portion of the parameter plane that is left out by the union of these three regions, is a one-dimensional curve, on which the function $H_{β,h,p}$ has a stationary inflection point. Finding out the exact order of the mixing time on this curve remains an open question. Finally, we show that if $H_{β,h,p}$ has multiple local maximizers (metastable states), then one can create a restricted version of the original Glauber dynamics, which still mixes in time $Θ(N\log N)$.