In this paper, we consider a continuous-time autoregressive fractionally integrated moving average (CARFIMA) model, which is defined as the stationary solution of a stochastic differential equation driven by a standard fractional Brownian motion. Like the discrete-time ARFIMA model, the CARFIMA model is useful for studying time series with short memory, long memory and antipersistence. We investigate the stationarity of the model and derive its covariance structure. In addition, we derive the spectral density function of a stationary CARFIMA process.