Several games that arise from graph coloring have been introduced and studied. Let $\varphi$ denote a graph invariant that arises from such a game. If $G$ is a graph and $\varphi(G-x)\neq \varphi(G)=k$, $k \geq 1$, holds true for every vertex $x \in V(G)$, then $G$ is called a $k$-$\varphi$-game-vertex-critical graph. We study the concept of $\varphi$-game-vertex-criticality for $\varphi \in \{χ_g, χ_i, χ_{ig}^{A}, χ_{ig}^{AB}\}$, where $χ_g$ denotes the standard game chromatic number, $χ_i$ denotes the indicated game chromatic number and $χ_{ig}^{A}$, $χ_{ig}^{AB}$ denote two versions of the independence game chromatic number. Since the game chromatic number $\varphi(G-x)$ can either decrease or increase with respect to $\varphi(G)$, we distinguish between lower, upper and mixed vertex-criticality. We show that for $\varphi \in \{χ_g, χ_{ig}^{A}, χ_{ig}^{AB}\}$ the difference $\varphi(G)-\varphi(G-x)$, $x \in V(G)$, can be arbitrarily large. A characterization of $2$-$\varphi$-game-vertex-critical and (connected) $3$-$\varphi$-lower-game-vertex-critical graphs for all $\varphi \in \{χ_g, χ_i, χ_{ig}^{A}, χ_{ig}^{AB}\}$ is given. It is shown that $χ_g$-game-vertex-critical, $χ_{ig}^{A}$-game-vertex-critical and $χ_{ig}^{AB}$-game-vertex-critical graphs are not necessarily connected. However, it is also shown that $χ_i$-lower-game-vertex-critical graphs are always connected.