In the context of physical layer security, a physical layer characteristic is used as a common source of randomness to generate the secret key. Therefore an accurate estimation of this characteristic is the core for reliable secret key generation. Estimation of almost all the existing physical layer characteristic suffer dramatically at low signal to noise (SNR) levels. In this paper, we propose a novel secret key generation algorithm that is based on the estimated angle of arrival (AoA) between the two legitimate nodes. Our algorithm has an outstanding performance at very low SNR levels. Our algorithm can exploit either the Azimuth AoA to generate the secret key or both the Azimuth and Elevation angles to generate the secret key. Exploiting a second common source of randomness adds an extra degree of freedom to the performance of our algorithm. We compare the performance of our algorithm to the algorithm that uses the most commonly used characteristics of the physical layer which are channel amplitude and phase. We show that our algorithm has a very low bit mismatch rate (BMR) at very low SNR when both channel amplitude and phase based algorithm fail to achieve an acceptable BMR.