We obtain limit theorems for a class of nonlinear discrete-time processes $X(n)$ called the $k$-th order Volterra processes of order $k$. These are moving average $k$-th order polynomial forms: \[ X(n)=\sum_{0<i_1,\ldots,i_k<\infty}a(i_1,\ldots,i_k)ε_{n-i_1}\ldotsε_{n-i_k}, \] where $\{ε_i\}$ is i.i.d.\ with $\mathbb{E} ε_i=0$, $\mathbb{E} ε_i^2=1$, where $a(\cdot)$ is a nonrandom coefficient, and where the diagonals are included in the summation. We specify conditions for $X(n)$ to be well-defined in $L^2(Ω)$, and focus on central and non-central limit theorems. We show that normalized partial sums of centered $X(n)$ obey the central limit theorem if $a(\cdot)$ decays fast enough so that $X(n)$ has short memory. We prove a non-central limit theorem if, on the other hand, $a(\cdot)$ is asymptotically some slowly decaying homogeneous function so that $X(n)$ has long memory. In the non-central case the limit is a linear combination of Hermite-type processes of different orders. This linear combination can be expressed as a centered multiple Wiener-Stratonovich integral.