It is anticipated that aerial-terrestrial integrated networks incorporating unmanned aerial vehicles (UAVs) mounted relays will offer improved coverage and connectivity in the beyond 5G era. Meanwhile, federated learning (FL) is a promising distributed machine learning technique for building inference models over wireless networks due to its ability to maintain user privacy and reduce communication overhead. However, off-the-shelf FL models aggregate global parameters at a central parameter server (CPS), increasing energy consumption and latency, as well as inefficiently utilizing radio resource blocks (RRBs) for distributed user devices (UDs). This paper presents a resource-efficient FL framework, called FedMoD (\textbf{fed}erated learning with \textbf{mo}del \textbf{d}issemination), for millimeter-wave (mmWave) aerial-terrestrial integrated networks with the following two unique characteristics. Firstly, FedMoD presents a novel decentralized model dissemination algorithm that makes use of UAVs as local model aggregators through UAV-to-UAV and device-to-device (D2D) communications. As a result, FedMoD (i) increases the number of participant UDs in developing FL model and (ii) achieves global model aggregation without involving CPS. Secondly, FedMoD reduces the energy consumption of FL using radio resource management (RRM) under the constraints of over-the-air learning latency. In order to achieve this, by leveraging graph theory, FedMoD optimizes the scheduling of line-of-sight (LOS) UDs to suitable UAVs/RRBs over mmWave links and non-LOS UDs to available LOS UDs via overlay D2D communications. Extensive simulations reveal that decentralized FedMoD offers same convergence rate performance as compared to conventional FL frameworks.