The COVID-19 pandemic is an unprecedented public health crisis with broad social and economic consequences. We conducted four surveys between April and August 2020 using the graph-based open-ended survey (GOS) framework, and investigated the most pressing concerns and issues for the general public in Japan. The GOS framework is a hybrid of the two traditional survey frameworks that allows respondents to post their opinions in a free-format style, which can subsequently serve as one of the choice items for other respondents, just as in a multiple-choice survey. As a result, this framework generates an opinion graph that relates opinions and respondents. We can also construct annotated opinion graphs to achieve a higher resolution. By clustering the annotated opinion graphs, we revealed the characteristic evolution of the response patterns as well as the interconnectedness and multi-faceted nature of opinions. Substantively, our notable finding is that "social pressure," not "infection risk," was one of the major concerns of our respondents. Social pressure refers to criticism and discrimination that they anticipate receiving from others should they contract COVID-19. It is possible that the collectivist nature of Japanese culture coupled with the government's policy of relying on personal responsibility to combat COVID-19 explains some of the above findings, as the latter has led to the emergence of vigilantes. The presence of mutual surveillance can contribute to growing skepticism toward others as well as fear of ostracism, which may have negative consequences at both the societal and individual levels.