In recent years, Semantic Communication (SemCom), which aims to achieve efficient and reliable transmission of meaning between agents, has garnered significant attention from both academia and industry. To ensure the security of communication systems, encryption techniques are employed to safeguard confidentiality and integrity. However, existing encryption schemes encounter obstacles when applied to SemCom. To address this issue, this paper explores the feasibility of applying homomorphic encryption (HE) to SemCom. Initially, we review the encryption algorithms utilized in mobile communication systems and analyze the challenges associated with their application to SemCom. Subsequently, we overview HE techniques and employ scale-invariant feature transform (SIFT) to demonstrate that the extractable semantic information can be preserved in homomorphic encrypted ciphertext. Based on this finding, we further propose the HE-joint source-channel coding (HE-JSCC) scheme, where the traditional JSCC model architecture is modified to support HE operations. Moreover, we present the simulation results for image classification and image generation tasks. Furthermore, we provide potential future research directions for homomorphic encrypted SemCom.