During the COVID-19 pandemic, "Chinese Virus" emerged as a controversial term for coronavirus. To some, it may seem like a neutral term referring to the physical origin of the virus. To many others, however, the term is in fact attaching ethnicity to the virus. While both arguments appear reasonable, quantitative analysis of the term's real-world usage is lacking to shed light on the issues behind the controversy. In this paper, we attempt to fill this gap. To model the substantive difference of tweets with controversial terms and those with non-controversial terms, we apply topic modeling and LIWC-based sentiment analysis. To test whether "Chinese Virus" and "COVID-19" are interchangeable, we formulate it as a classification task, mask out these terms, and classify them using the state-of-the-art transformer models. Our experiments consistently show that the term "Chinese Virus" is associated with different substantive topics and sentiment compared with "COVID-19" and that the two terms are easily distinguishable by looking at their context.