This paper proposes feasible asymptotically efficient estimators for a certain class of Gaussian noises with self-similar and stationary properties, which includes the fractional Gaussian noise, under high frequency observations. In this setting, the optimal rate of estimation depends on whether either the Hurst or diffusion parameters is known or not. This is due to the singularity of the asymptotic Fisher information matrix for simultaneous estimation of the above two parameters. One of our key ideas is to extend the Whittle estimation method to the situation of high frequency observations. We show that our estimators are asymptotically efficient in Fisher's sense.