In this paper, we study the path planning for a cellular-connected unmanned aerial vehicle (UAV) to minimize its flying distance from given initial to final locations, while ensuring a target link quality in terms of the large-scale channel gain with each of its associated ground base stations (GBSs) during the flight. To this end, we propose the use of radio map that provides the information on the large-scale channel gains between each GBS and uniformly sampled locations on a three-dimensional (3D) grid over the region of interest, which are assumed to be time-invariant due to the generally static and large-size obstacles therein (e.g., buildings). Based on the given radio maps of the GBSs, we first obtain the optimal UAV path by solving an equivalent shortest path problem (SPP) in graph theory. To reduce the computation complexity of the optimal solution, we further propose a grid quantization method whereby the grid points in each GBS's radio map are more coarsely sampled by exploiting the spatial channel correlation over neighboring grids. Then, we solve the approximate SPP over the reduced-size radio map (graph) more efficiently. Numerical results show that the proposed solutions can effectively minimize the flying distance of the UAV subject to its communication quality constraint. Moreover, a flexible trade-off between performance and complexity can be achieved by adjusting the quantization ratio for the radio map.