It has been one year since the outbreak of the COVID-19 pandemic. The good news is that vaccines developed by several manufacturers are being actively distributed worldwide. However, as more and more vaccines become available to the public, various concerns related to vaccines become the primary barriers that may hinder the public from getting vaccinated. Considering the complexities of these concerns and their potential hazards, this study aims to offer a clear understanding about different population groups' underlying concerns when they talk about COVID-19 vaccines, particular those active on Reddit. The goal is achieved by applying LDA and LIWC to characterizing the pertaining discourse with insights generated through a combination of quantitative and qualitative comparisons. Findings include: 1) during the pandemic, the proportion of Reddit comments predominated by conspiracy theories outweighed that of any other topics; 2) each subreddit has its own user bases, so information posted in one subreddit may not reach those from other subreddits; 3) since users' concerns vary across time and subreddits, communication strategies must be adjusted according to specific needs. The results of this study manifest challenges as well as opportunities in the process of designing effective communication and immunization programs.