Supporting reliable and seamless mobility for aerial users, such as unmanned aerial vehicles (UAVs), is a key challenge for the next-generation cellular systems. To tackle this challenge, we propose a joint beamforming, user association, and UAV-height control framework for cellular-connected multi-UAV networks with multiple antenna base stations (BSs). With the aim of maximizing the minimum achievable rate for UAVs subject to co-existed terrestrial user equipment's rate constraints, we devise a hierarchical bi-layer iterative algorithm to optimize BSs' beamforming vectors, UAV association matrix, and the height of UAVs jointly. With the aid of projection gradient method in inner layer iteration and geometric program modelling plus convex-concave procedure in outer layer iteration, our proposed algorithm is proved to converge to a local optimum. Taking mobility characteristics of UAVs into account, we also exploit our proposed algorithm for imperfect channel estimation scenario. Numerical results show that our proposed algorithm can achieve improved UAVs' minimum achievable rate compared with that of the conventional nearest association of UAVs for both perfect and imperfect channel estimation scenarios. Moreover, we also examine the trade-off between the UAVs' minimum achievable rate and the frequency for updating optimization variables with single moving UAV.