This paper establishes the capacity of additive white Gaussian noise (AWGN) channels with phase-quantized output. We show that a rotated $2^b$-phase shift keying scheme is the capacity-achieving input distribution for a complex AWGN channel with $b$-bit phase quantization. The result is then used to establish the expression for the channel capacity as a function of average power constraint $P$ and quantization bits $b$. The outage performance of phase-quantized system is also investigated for the case of Rayleigh fading when the channel state information (CSI) is only known at the receiver. Our findings suggest the existence of a threshold in the rate $R$, above which the outage exponent of the outage probability changes abruptly. In fact, this threshold effect in the outage exponent causes $2^b$-PSK to have suboptimal outage performance at high SNR.