We discuss the shuffle product of the Schur multiple zeta values, which are the special values of Schur multiple zeta functions. We first define $2$-labeled Schur posets to generalize Yamamoto's integral expression of the multiple zeta values and consider the product of hook-type Schur multiple zeta values by using these posets. Then, for the derived terms, we introduce a modified Hurwitz-type Schur multiple zeta function of hook type, named an elementary factorial Schur multiple zeta function. Furthermore, we generalize $2$-labeled Schur posets to consider the shuffle product of the elementary factorial Schur multiple zeta values and obtain an explicit formula for their shuffle product.