This work is motivated by the long-standing open problem of designing asymptotically order-optimal aperiodic polyphase sequence sets with respect to the celebrated Welch bound. Attempts were made by Mow over 30 years ago, but a comprehensive understanding to this problem is lacking. Our first key contribution is an explicit upper bound of generalized quadratic Gauss sums which is obtained by recursively applying Paris' asymptotic expansion and then bounding it by leveraging the fast convergence property of the Fibonacci zeta function. Building upon this major finding, our second key contribution includes four systematic constructions of order-optimal sequence sets with low aperiodic correlation and/or ambiguity properties via carefully selected Chu sequences and Alltop sequences. For the first time in the literature, we reveal that the full Alltop sequence set is asymptotically optimal for its low aperiodic correlation sidelobes. Besides, we introduce a novel subset of Alltop sequences possessing both order-optimal aperiodic correlation and ambiguity properties for the entire time-shift window.