Wireless channel foundation model (WCFM) is a task-agnostic AI model that is pre-trained to learn a universal channel representation for a wide range of communications and sensing tasks. While existing works on WCFM have demonstrated its great potentials in various downstream tasks, the models are all trained using perfect (i.e., error-free and complete) channel information state (CSI) data. In practical systems, however, only degraded CSI obtained from pilot-based channel estimation is accessible, leading to distorted channel representations and performance degradation in downstream tasks for some real-world environments with severe noise and interference. To address this issue, this paper proposes a new paradigm for WCFM, termed as Filter-and-Attend. In this paradigm, Filter refers to explicitly suppressing noise-plus-interference (NPI) in the received signals, while Attend means performing correlation-aware CSI completion and feature extraction using attention mechanism. Specifically, an enhanced WCFM architecture is developed. In this architecture, coarse estimates of the CSIs are first obtained and exploited to construct two projection matrices that extract NPI components in the received signals, which are further processed and removed by a subtraction module. The filtered signal is subsequently passed through a CSI completion network to get a clean CSI for feature extraction. Simulation results demonstrated that compared to the state-of-the-art solutions, WCFM with NPI suppression structure achieves improved performance on various downstream tasks including time-domain channel prediction, frequency-domain channel prediction, and localization.