Can public social media data be harnessed to predict COVID-19 case counts? We analyzed approximately 15 million COVID-19 related posts on Weibo, a popular Twitter-like social media platform in China, from November 1, 2019 to March 31, 2020. We developed a machine learning classifier to identify "sick posts," which are reports of one's own and other people's symptoms and diagnosis related to COVID-19. We then modeled the predictive power of sick posts and other COVID-19 posts on daily case counts. We found that reports of symptoms and diagnosis of COVID-19 significantly predicted daily case counts, up to 14 days ahead of official statistics. But other COVID-19 posts did not have similar predictive power. For a subset of geotagged posts (3.10% of all retrieved posts), we found that the predictive pattern held true for both Hubei province and the rest of mainland China, regardless of unequal distribution of healthcare resources and outbreak timeline. Researchers and disease control agencies should pay close attention to the social media infosphere regarding COVID-19. On top of monitoring overall search and posting activities, it is crucial to sift through the contents and efficiently identify true signals from noise.