Automated wireless spectrum monitoring across frequency, time and space will be essential for many future applications. Manual and fine-grained spectrum analysis is becoming impossible because of the large number of measurement locations and complexity of the spectrum use landscape. Detecting unexpected behaviors in the wireless spectrum from the collected data is a crucial part of this automated monitoring, and the control of detected anomalies is a key functionality to enable interaction between the automated system and the end user. In this paper we look into the wireless spectrum anomaly detection problem for crowdsourced sensors. We first analyze in detail the nature of these anomalies and design effective algorithms to bring the higher dimensional input data to a common feature space across sensors. Anomalies can then be detected as outliers in this feature space. In addition, we investigate the importance of user feedback in the anomaly detection process to improve the performance of unsupervised anomaly detection. Furthermore, schemes for generalizing user feedback across sensors are also developed to close the anomaly detection loop.