The crystals for a finite-dimensional complex reductive Lie algebra $\mathfrak{g}$ encode the structure of its representations, yet can also reveal surprising new structure of their own. We study the cactus group $C_{\mathfrak{g}}$, constructed using the Dynkin diagram of $\mathfrak{g}$, and its combinatorial action on any $\mathfrak{g}$-crystal via Schützenberger involutions. We compare this action with that of the Berenstein-Kirillov group on Gelfand-Tsetlin patterns. Henriques and Kamnitzer define an action of $C_n=C_{\mathfrak{gl}_n}$ on $n$-tensor products of $\mathfrak{g}$-crystals, for any $\mathfrak{g}$ as above. We discuss the crystal corresponding to the $\mathfrak{gl}_n \times \mathfrak{gl}_m$-representation $Λ^N(\mathbb{C}^n \otimes \mathbb{C}^m),$ derive skew Howe duality on the crystal level and show that the two types of cactus group actions agree in this setting. A future application of this result is discussed in studying two families of maximal commutative subalgebras of the universal enveloping algebra, the shift of argument and Gaudin algebras, where an algebraically constructed monodromy action matches that of the cactus group.