Unmanned aerial vehicles (UAVs) have been increasingly utilized in various civilian and military applications such as remote sensing, border patrolling, disaster monitoring, and communication coverage extension. However, there are still prone to several cyber attacks such as GPS spoofing attacks, distributed denial-of-service (DDoS) attacks, and man-in-the-middle attacks to obtain their collected information or to enforce the UAVs to perform their requested actions which may damage the UAVs or their surrounding environment or even endanger the safety of human in the operation field. In this paper, we propose a trust monitoring mechanism in which a centralized unit (e.g. the ground station) regularly observe the behavior of the UAVs in terms of their motion path, their consumed energy, as well as the number of their completed tasks and measure a relative trust score for the UAVs to detect any abnormal behaviors in a real-time manner. Our simulation results show that the trust model can detect malicious UAVs, which can be under various cyber-security attacks such as flooding attacks, man-in-the-middle attacks, GPS spoofing attack in real-time.