Context: In-app advertising is the primary source of revenue for many mobile apps. The cost of advertising (ad cost) is non-negligible for app developers to ensure a good user experience and continuous profits. Previous studies mainly focus on addressing the hidden performance costs generated by ads, including consumption of memory, CPU, data traffic, and battery. However, there is no research onanalyzing users' perceptions of ads' performance costs to our knowledge. Objective: To fill this gap and better understand the effects of performance costs of in-app ads on user experience, we conduct a study on analyzing user concerns about ads' performance costs. Method: First, we propose RankMiner, an approach to quantify user concerns about specific appissues, including performance costs. Then, based on the usage traces of 20 subject apps, we measure the performance costs of ads. Finally, we conduct correlation analysis on the performance costs and quantified user concerns to explore whether users complain more for higher performance costs. Results: Our findings include the following: (1) RankMiner can quantify users' concerns better than baselines by an improvement of 214% and 2.5% in terms of Pearson correlation coefficient (a metricfor computing correlations between two variables) and NDCG score (a metric for computing accuracyin prioritizing issues), respectively. (2) The performance costs of the with-ads versions are statistically significantly larger than those of no-ads versions with negligible effect size; (3) Users are moreconcerned about the battery costs of ads, and tend to be insensitive to ads' data traffic costs. Conclusion: Our study is complementary to previous work on in-app ads, and can encourage devel-opers to pay more attention to alleviating the most user-concerned performance costs, such as battery cost.