Overlapping community detection (OCD) is a fundamental graph data analysis task for extracting graph patterns. Traditional OCD methods can be broadly divided into node clustering and link clustering approaches, both of which rely solely on link information to identify overlapping communities. In recent years, deep learning-based methods have made significant advancements for this task. However, existing GNN-based approaches often face difficulties in effectively integrating link, attribute, and prior information, along with challenges like limited receptive fields and over-smoothing, which hinder their performance on complex overlapping community detection. In this paper, we propose a Weak-clique based Overlapping Community Detection method, namely WOCD, which incorporates prior information and optimizes the use of link information to improve detection accuracy. Specifically, we introduce pseudo-labels within a semi-supervised framework to strengthen the generalization ability, making WOCD more versatile. Furthermore, we initialize pseudo-labels using weak cliques to fully leverage link and prior information, leading to better detection accuracy. Additionally, we employ a single-layer Graph Transformer combined with GNN, which achieves significant performance improvements while maintaining efficiency. We evaluate WOCD on eight real-world attributed datasets, and the results demonstrate that it outperforms the state-of-the-art semi-supervised OCD method by a significant margin in terms of accuracy.