Semantic communication aims to facilitate purposeful information exchange among diverse intelligent entities, including humans, machines, and organisms. It emphasizes precise semantic transmission over data fidelity, striving for meaningful expression while optimizing communication resources for efficient information transfer. Nevertheless, extant semantic communication systems face security, privacy, and trust challenges in integrating AI technologies for intelligent communication applications. This paper presents a comprehensive survey of security and privacy threats across various layers of semantic communication systems and discusses state-of-the-art countermeasures within both academic and industry contexts. Finally, we identify critical open issues in this burgeoning field warranting further investigation.