Consider a graph $Γ$. A set $ S $ of vertices in $Γ$ is called a {cyclic vertex cutset} of $Γ$ if $Γ- S$ is disconnected and has at least two components containing cycles. If $Γ$ has a cyclic vertex cutset, then it is said to be {cyclically separable}. The {cyclic vertex connectivity} is the minimum cardinality of a cyclic vertex cutset of $Γ$. The power graph $\mathcal{P}(G)$ of a group $G$ is the undirected simple graph with vertex set $G$ and two distinct vertices are adjacent if one of them is a positive power of the other. If $G$ is a cyclic, dihedral, or dicyclic group, we determine the order of $G$ such that $\mathcal{P}(G)$ is cyclically separable. Then we characterize the equality of vertex connectivity and cyclic vertex connectivity of $\mathcal{P}(G)$ in terms of the order of $G$.