Recently, Gillespie, Levinson and Purbhoo introduced a crystal-like structure for shifted tableaux, called the shifted tableau crystal. We introduce a shifted analogue of the crystal reflection operators, which coincides with the restriction of the shifted Schützenberger involution to any primed interval of two adjacent letters. Unlike type $A$ Young tableau crystals, these operators do not realize an action of the symmetric group on the shifted tableau crystal because braid relations do not hold. We exhibit a natural internal action of the cactus group, realized by restrictions of the shifted Schützenberger involution on primed intervals of the underlying crystal alphabet.