We study the extreme point process associated to the off-diagonal components in the matrix representation of the Gaussian $β$-Ensemble and prove its convergence to Poisson point process as $n\to +\infty$ when the inverse temperature $β$ scales with $n$ and tends to $0$. We consider two main high temperature regimes: $\displaystyle{β\ll \frac{1}{n}}$ and $\displaystyle{nβ= 2γ\geq 0}$. The normalizing sequences are explicitly given in each cases. As a consequence, we estimate the first order asymptotic of the largest eigenvalue of the Gaussian $β$-Ensemble.