Space-Air-Ground Integrated Networks (SAGINs) are pivotal for enabling ubiquitous connectivity in 6G systems, yet they face significant challenges due to severe satellite-to-ground link impairments. Although Unmanned Aerial Vehicles (UAVs) can function as relay nodes to compensate for air-to-ground channel degradation, the satellite-to-UAV link remains a critical bottleneck. Semantic Communication (SemCom) emerges as a promising solution to enhance spectral efficiency by transmitting essential semantic information. This paper proposes a novel multi-cluster UAV-aided SAGIN SemCom architecture that supports both semantic users (SemUsers) and conventional users (ConUsers). While SemCom is employed in the satellite-to-UAV link to improve transmission efficiency, the UAVs implement an intelligent adaptive relay strategy, capable of either directly forwarding semantic data to SemUsers or converting it into bit-level data for ConUsers. Compared to existing similar schemes, this design guarantees the high-efficiency advantages of SemCom while enabling network access for larger coverage area. A joint optimization problem is formulated to maximize the system's sum-rate through coordinated allocation of power, bandwidth, and UAV positions. To address this non-convex problem, we develop an efficient alternating optimization (AO) algorithm, which decomposes the original problem into tractable subproblems. Numerical results demonstrate that the proposed algorithm significantly outperforms baseline schemes in terms of both sum-rate and spectral efficiency across various channel conditions and user distributions, underscoring the importance of joint resource allocation and intelligent UAV deployment.