The popularity of 5G networks poses a huge challenge for malicious traffic detection technology. The reason for this is that as the use of 5G technology increases, so does the risk of malicious traffic activity on 5G networks. Malicious traffic activity in 5G networks not only has the potential to disrupt communication services, but also to compromise sensitive data. This can have serious consequences for individuals and organizations. In this paper, we first provide an in-depth study of 5G technology and 5G security. Next we analyze and discuss the latest malicious traffic detection under AI and their applicability to 5G networks, and compare the various traffic detection aspects addressed by SOTA. The SOTA in 5G traffic detection is also analyzed. Next, we propose seven criteria for traffic monitoring datasets to confirm their suitability for future traffic detection studies. Finally, we present three major issues that need to be addressed for traffic detection in 5G environment. The concept of incremental learning techniques is proposed and applied in the experiments, and the experimental results prove to be able to solve the three problems to some extent.