Sharing the spectrum with in-band full-duplex (FD) primary users (PU) is a challenging and interesting problem in the underlay cognitive radio (CR) systems. The self-interference introduced at the primary network may dramatically impede the secondary user (SU) opportunity to access the spectrum. To tackle this problem, we use the so-called improper Gaussian signaling. Particularly, we assume a system with a SU pair working in a half-duplex mode that uses improper Gaussian signaling while the FD PU pair implements the regular proper Gaussian signaling. First, we derive a closed form expression and an upper bound for the SU and PU outage probabilities, respectively. Second, we optimize the SU signal parameters to minimize its outage probability while maintaining the required PU quality-of-service based on the average channel state information (CSI). Moreover, we provide the conditions to reap merits from employing improper Gaussian signaling at the SU. Third, we design the SU signal parameters based on perfect knowledge of its direct link instantaneous CSI and investigate all benefits that can be achieved at both the SU and PU. Finally, we provide some numerical results that demonstrate the advantages of using improper Gaussian signaling to access the spectrum of the FD PU.