Multiple carrier frequency offsets (CFOs) present in the uplink of orthogonal frequency division multiple access (OFDMA) systems adversely affect subcarrier orthogonality and impose a serious performance loss. In this paper, we propose the application of time domain receiver windowing to concentrate the leakage caused by CFOs to a few adjacent subcarriers with almost no additional computational complexity. This allows us to approximate the interference matrix with a quasi-banded matrix by neglecting small elements outside a certain band which enables robust and computationally efficient signal detection. The proposed CFO compensation technique is applicable to all types of subcarrier assignment techniques. Simulation results show that the quasi-banded approximation of the interference matrix is accurate enough to provide almost the same bit error rate performance as that of the optimal solution. The excellent performance of our proposed method is also proven through running an experiment using our FPGA-based system setup.