Recently, Gillespie, Levinson and Purbhoo introduced a crystal-like structure for shifted tableaux, called the shifted tableau crystal. We introduce, on this structure, a shifted version of the crystal reflection operators, which coincide with the restrictions of the shifted Schützenberger involution to any primed interval of two adjacent letters. Unlike type $A$ Young tableau crystals, these operators do not realize an action of the symmetric group on the shifted tableau crystal since the braid relations do not need to hold. Following a similar approach as Halacheva, we exhibit a natural internal action of the cactus group on this crystal, realized by the restrictions of the shifted Schützenberger involution to all primed intervals of the underlying crystal alphabet, containing, in particular, the aforesaid action of the shifted crystal reflection operator analogues.