Conversational Agents (CAs, chatbots) are systems with the ability to interact with users using natural human dialogue. While much of the research on CAs for sexual health has focused on adult populations, the insights from such research may not apply to CAs for youth. The study aimed to comprehensively evaluate the state-of-the-art research on sexual health CAs for youth. Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we synthesized peer-reviewed studies specific to sexual health CAs designed for youth over the past 14 years. We found that most sexual health CAs were designed to adopt the persona of health professionals to provide general sexual and reproductive health information for youth. Text was the primary communication mode in all sexual health CAs, with half supporting multimedia output. Many sexual health CAs employed rule-based techniques to deliver pre-written expert knowledge on sexual health; yet most sexual health CAs did not have the safety features in place. While youth appreciated accessibility to non-judgmental and confidential conversations about sexual health topics, they perceived current sexual health CAs provided limited sexual health information that is not inclusive of sexual and/or gender minorities. Our review brings to light sexual health CAs needing further development and evaluation and we identify multiple important areas for future work. While the new trend of large language models (LLMs) based CAs can make such technologies more feasible, the privacy and safety of the systems should be prioritized. Finally, best practices for risk mitigation and ethical development of sexual health CAs with and for youth are needed.