The integration of unmanned aerial vehicles (UAVs) and millimeter wave (mmWave) wireless systems has been recently proposed to provide high data rate aerial links for next generation wireless networks. However, establishing UAV-based mmWave links is quite challenging due to the random fluctuations of hovering UAVs which can induce antenna gain mismatch between transmitter and receiver. To assess the benefit of UAV-based mmWave links, in this paper, tractable, closed-form statistical channel models are derived for three UAV communication scenarios: (i) a direct UAV-to-UAV link, (ii) an aerial relay link in which source, relay, and destination are hovering UAVs, and (iii) a relay link in which a hovering UAV connects a ground source to a ground destination. The accuracy of the derived analytical expressions is corroborated by performing Monte-Carlo simulations. Numerical results are then used to study the effect of antenna directivity gain under different channel conditions for establishing reliable UAV-based mmWave links in terms of achieving minimum outage probability. It is shown that the performance of such links is largely dependent on the random fluctuations of hovering UAVs. Moreover, higher antenna directivity gains achieve better performance at low SNR regime. Nevertheless, at the high SNR regime, lower antenna directivity gains result in a more reliable communication link. The developed results can therefore be applied as a benchmark for finding the optimal antenna directivity gain of UAVs under the different levels of instability without resorting to time-consuming simulations.