The intrinsic factor that drives the human movement remains unclear for decades. While our observations from intra-urban and inter-urban trips both demonstrate a universal law in human mobility. Be specific, the probability from one location to another is inversely proportional to the number of population living in locations which are closer than the destination. A simple rank-based model is then presented, which is parameterless but predicts human flows with a convincing fidelity. Besides, comparison with other models shows that our model is more stable and fundamental at different spatial scales by implying the strong correlation between human mobility and social relationship.