To support high data rate urgent or ad hoc communications, we consider mmWave UAV cellular networks and the associated challenges and solutions. To enable fast beamforming training and tracking, we first investigate a hierarchical structure of beamforming codebooks and design of hierarchical codebooks with different beam widths via the sub-array techniques. We next examine the Doppler effect as a result of UAV movement and find that the Doppler effect may not be catastrophic when high gain directional transmission is used. We further explore the use of millimeter wave spatial division multiple access and demonstrate its clear advantage in improving the cellular network capacity. We also explore different ways of dealing with signal blockage and point out that possible adaptive UAV cruising algorithms would be necessary to counteract signal blockage. Finally, we identify a close relationship between UAV positioning and directional millimeter wave user discovery, where update of the former may directly impact the latter and vice versa.