This paper presents how to leak private information from a wireless signal classifier by launching an over-the-air membership inference attack (MIA). As machine learning (ML) algorithms are used to process wireless signals to make decisions such as PHY-layer authentication, the training data characteristics (e.g., device-level information) and the environment conditions (e.g., channel information) under which the data is collected may leak to the ML model. As a privacy threat, the adversary can use this leaked information to exploit vulnerabilities of the ML model following an adversarial ML approach. In this paper, the MIA is launched against a deep learning-based classifier that uses waveform, device, and channel characteristics (power and phase shifts) in the received signals for RF fingerprinting. By observing the spectrum, the adversary builds first a surrogate classifier and then an inference model to determine whether a signal of interest has been used in the training data of the receiver (e.g., a service provider). The signal of interest can then be associated with particular device and channel characteristics to launch subsequent attacks. The probability of attack success is high (more than 88% depending on waveform and channel conditions) in identifying signals of interest (and potentially the device and channel information) used to build a target classifier. These results show that wireless signal classifiers are vulnerable to privacy threats due to the over-the-air information leakage of their ML models