Detecting communities in complex networks can shed light on the essential characteristics and functions of the modeled phenomena. This topic has attracted researchers of various fields from both academia and industry. Among the different methods implemented for community detection, Genetic Algorithms (GA) have become popular recently. Considering the drawbacks of the currently used locus-based and solution-vector-based encodings to represent the individuals, in this paper, we propose (1) a new node similarity-based encoding method to represent a network partition as an individual named MST-based. Then, we propose (2) a new Adaptive Genetic Algorithm for Community Detection, along with (3) a new initial population generation function, and (4) a new adaptive mutation function called sine-based mutation function. Using the proposed method, we combine similarity-based and modularity-optimization-based approaches to find the communities of complex networks in an evolutionary framework. Besides the fact that the proposed representation scheme can avoid meaningless mutations or disconnected communities, we show that the new initial population generation function, and the new adaptive mutation function, can improve the convergence time of the algorithm. Experiments and statistical tests verify the effectiveness of the proposed method compared with several classic and state-of-the-art algorithms.