Unmanned aerial vehicles (UAVs) are increasingly being integrated into next-generation networks to enhance communication coverage and network capacity. However, the dynamic and mobile nature of UAVs poses significant security challenges, including jamming, eavesdropping, and cyber-attacks. To address these security challenges, this paper proposes a federated learning-based lightweight network with zero trust for enhancing the security of UAV networks. A novel lightweight spectrogram network is proposed for UAV authentication and rejection, which can effectively authenticate and reject UAVs based on spectrograms. Experiments highlight LSNet's superior performance in identifying both known and unknown UAV classes, demonstrating significant improvements over existing benchmarks in terms of accuracy, model compactness, and storage requirements. Notably, LSNet achieves an accuracy of over $80\%$ for known UAV types and an Area Under the Receiver Operating Characteristic (AUROC) of $0.7$ for unknown types when trained with all five clients. Further analyses explore the impact of varying the number of clients and the presence of unknown UAVs, reinforcing the practical applicability and effectiveness of our proposed framework in real-world FL scenarios.