Let $(X_i)_{i\geq 1}$ be a stationary mean-zero Gaussian process with covariances $ρ(k)=\PE(X_{1}X_{k+1})$ satisfying: $ρ(0)=1$ and $ρ(k)=k^{-D} L(k)$ where $D$ is in $(0,1)$ and $L$ is slowly varying at infinity. Consider the $U$-process $\{U_n(r),\; r\in I\}$ defined as $$ U_n(r)=\frac{1}{n(n-1)}\sum_{1\leq i\neq j\leq n}\1_{\{G(X_i,X_j)\leq r\}}\; , $$ where $I$ is an interval included in $\rset$ and $G$ is a symmetric function. In this paper, we provide central and non-central limit theorems for $U_n$. They are used to derive the asymptotic behavior of the Hodges-Lehmann estimator, the Wilcoxon-signed rank statistic, the sample correlation integral and an associated scale estimator. The limiting distributions are expressed through multiple Wiener-Itô integrals.