PhaseLift is a noted convex optimization technique for phase retrieval that can recover a signal exactly from amplitude measurements only, with high probability. Conventional PhaseLift requires a relatively large number of samples that sometimes can be costly to acquire. % to compensate for the missing phase information and achieve effective phase retrieval. This paper focuses on some practical applications where the signal of interest is composed of a few Vandermonde components, such as line spectra.A novel phase retrieval framework, namely ANM-PhaseLift, is developed that exploits the Vandermonde structure to alleviate the sampling requirements. Specifically, the atom set of amplitude-based quadratic measurements is identified, and atomic norm minimization (ANM) is introduced into PhaseLift to considerably reduce the number of measurements that are needed for accurate phase retrieval. The benefit of ANM-PhaseLift is particularly attractive in applications where the Vandermonde structure is presented, such as massive MIMO and radar imaging.