Mobile application marketplaces are responsible for vetting apps to identify and mitigate security risks. Current vetting processes are labor-intensive, relying on manual analysis by security professionals aided by semi-automated tools. To address this inefficiency, we propose Mars, a system that leverages Large Language Models (LLMs) for automated risk identification and profiling. Mars is designed to concurrently analyze multiple applications across diverse risk categories with minimal human intervention. To enhance analytical precision and operational efficiency, Mars leverages a pre-constructed risk identification tree to extract relevant indicators from high-dimensional application features. This initial step filters the data, reducing the input volume for the LLM and mitigating the potential for model hallucination induced by irrelevant features. The extracted indicators are then subjected to LLM analysis for final risk determination. Furthermore, Mars automatically generates a comprehensive evidence chain for each assessment, documenting the analytical process to provide transparent justification. These chains are designed to facilitate subsequent manual review and to inform enforcement decisions, such as application delisting. The performance of Mars was evaluated on a real-world dataset from a partner Android marketplace. The results demonstrate that Mars attained an F1-score of 0.838 in risk identification and an F1-score of 0.934 in evidence retrieval. To assess its practical applicability, a user study involving 20 expert analysts was conducted, which indicated that Mars yielded a substantial efficiency gain, ranging from 60% to 90%, over conventional manual analysis.