Conventional methods for diagnosing Social Anxiety Disorder (SAD), such as clinical interviews and self-reported questionnaires, often face accessibility barriers and subjective biases, underscoring the need for objective physiological markers. This study investigates heart rate (HR) and heart rate variability (HRV) as potential indicators of SAD by analyzing cardiovascular responses to anxiety-inducing speech tasks across four distinct phases: baseline, anticipation, speech activity, and reflection. In a controlled laboratory setting, we analyzed data from 51 participants and found that HRV decreased and HR increased during the anticipation and speech activity phases compared to baseline, while the reflection phase showed a reversal, with HRV increasing and HR decreasing. Participants with SAD exhibited lower HRV, higher HR, and greater self-reported anxiety than non-SAD participants across all phases. These findings enhance our understanding of the physiological signatures of social anxiety and have implications for developing wearable-based monitoring systems for SAD detection and intervention. To support further research, we also release a dataset capturing multi-phase anxiety responses, advancing physiological-based mental health assessment