This paper presents a dual-domain low-complexity expectation propagation (EP) detection framework for affine frequency division multiplexing (AFDM) systems. By analyzing the structural properties of the effective channel matrices in both the time and affine frequency (AF) domains, our key observation is the domain-specific quasi-banded sparsity patterns, including AF-domain sparsity under frequency-selective channels and time-domain sparsity under doubly-selective channels. Based on these observations, we develop an AF-domain EP (EP-AF) detector for frequency-selective channels and a time-domain EP (EP-T) detector for doubly-selective channels, respectively. By performing iterative inference in the time domain using the Gaussian approximation, the proposed EP-T detector avoids inverting the dense channel matrix in the AF domain. Furthermore, the proposed EP-AF and EP-T detectors leverage the aforementioned quasi-banded sparsity of the AF domain and time domain channel matrices, respectively, to reduce the complexity of matrix inversion from cubic to linear order. Simulation results demonstrate that the proposed low-complexity EP-AF detector achieves nearly identical error rate performance to its conventional counterpart, while the proposed low-complexity EP-T detector offers an attractive trade-off between detection performance and complexity.