High altitude platform stations (HAPS) have recently emerged as a new key stratospheric player in non-terrestrial networks (NTN) alongside satellites and low-altitude platforms. In this paper, we present the main communication links between HAPS and other NTN platforms, their advantages, and their challenges. Then, prospective network architectures in which HAPS plays an indispensable role in the future NTNs are presented such as ad-hoc, cell-free, and integrated access and backhaul. To showcase the importance of HAPS in the NTN, we provide comprehensive performance insights when using HAPS in the prospective architectures with the most suitable communication link. The insights show the HAPS' ability to interconnect the NTN nexus as well as their versatility by incorporating different metrics into the analysis such as routing latency, energy efficiency, coverage probability, and channel capacity. Depending on the architecture, HAPS will play different roles in NTN, such as a UAV network center, satellite relay, and ground network extension. Finally, the performance gain provided by HAPS usage in NTN is further highlighted by comparing the results when no HAPS are used.