Consider a generic random element $F_\infty= \sum_{\text{finite}} λ_k (N^2_k -1)$ in the second Wiener chaos with a finite number of non-zero coefficients in the spectral representation where $(N_k)_{k \ge 1}$ is a sequence of i.i.d $\mathscr{N}(0,1)$. Using the recently discovered (see Arras et al. \cite{a-a-p-s-stein}) stein operator $\RR_\infty$ associated to $F_\infty$, we introduce a new class of polynomials $$\PP_\infty:= \{ P_n = \RR^n_\infty \textbf{1} \, : \, n \ge 1 \}.$$ We analysis in details the case where $F_\infty$ is distributed as the normal product distribution $N_1 \times N_2$, and relate the associated polynomials class to Rota's {\it Umbral calculus} by showing that it is a \textit{Sheffer family} and enjoys many interesting properties. Lastly, we study the connection between the polynomial class $\PP_\infty$ and the non-central probabilistic limit theorems within the second Wiener chaos.