This paper discusses asymptotic distributions of various estimators of the underlying parameters in some regression models with long memory (LM) Gaussian design and nonparametric heteroscedastic LM moving average errors. In the simple linear regression model, the first-order asymptotic distribution of the least square estimator of the slope parameter is observed to be degenerate. However, in the second order, this estimator is $n^{1/2}$-consistent and asymptotically normal for $h+H<3/2$; nonnormal otherwise, where $h$ and $H$ are LM parameters of design and error processes, respectively. The finite-dimensional asymptotic distributions of a class of kernel type estimators of the conditional variance function $σ^2(x)$ in a more general heteroscedastic regression model are found to be normal whenever $H<(1+h)/2$, and non-normal otherwise. In addition, in this general model, $\log(n)$-consistency of the local Whittle estimator of $H$ based on pseudo residuals and consistency of a cross validation type estimator of $σ^2(x)$ are established. All of these findings are then used to propose a lack-of-fit test of a parametric regression model, with an application to some currency exchange rate data which exhibit LM.