Here we present a theoretical study on the main properties of Fractionally Integrated Exponential Generalized Autoregressive Conditional Heteroskedastic (FIEGARCH) processes. We analyze the conditions for the existence, the invertibility, the stationarity and the ergodicity of these processes. We prove that, if $\{X_t\}_{t \in \mathds{Z}}$ is a FIEGARCH$(p,d,q)$ process then, under mild conditions, $\{\ln(X_t^2)\}_{t\in\mathds{Z}}$ is an ARFIMA$(q,d,0)$, that is, an autoregressive fractionally integrated moving average process. The convergence order for the polynomial coefficients that describes the volatility is presented and results related to the spectral representation and to the covariance structure of both processes $\{\ln(X_t^2)\}_{t\in\mathds{Z}}$ and $\ {\ln(σ_t^2)\}_{t\in\mathds{Z}}$ are also discussed. Expressions for the kurtosis and the asymmetry measures for any stationary FIEGARCH$(p,d,q)$ process are also derived. The $h$-step ahead forecast for the processes $\{X_t\}_{t \in \mathds{Z}}$, $\{\ln(σ_t^2)\}_{t\in\mathds{Z}}$ and $\{\ln(X_t^2)\}_{t\in\mathds{Z}}$ are given with their respective mean square error forecast. The work also presents a Monte Carlo simulation study showing how to generate, estimate and forecast based on six different FIEGARCH models. The forecasting performance of six models belonging to the class of autoregressive conditional heteroskedastic models (namely, ARCH-type models) and radial basis models is compared through an empirical application to Brazilian stock market exchange index.