High-Performance Computing (HPC) in the public cloud democratizes the supercomputing power that most users cannot afford to purchase and maintain. Researchers have studied its viability, performance, and usability. However, HPC in the cloud has a unique feature -- users have to export data and computation to somewhat untrusted cloud platforms. Users will either fully trust cloud providers to protect from all kinds of attacks or keep sensitive assets in-house instead. With the recent deployment of the Trusted Execution Environment (TEE) in the cloud, confidential computing for HPC in the cloud is becoming practical for addressing users' privacy concerns. This paper discusses the threat models, unique challenges, possible solutions, and significant gaps, focusing on TEE-based confidential HPC computing. We hope this discussion will improve the understanding of this new topic for HPC in the cloud and promote new research directions.