In recent years, the amalgamation of satellite communications and aerial platforms into space-air-ground integrated network (SAGINs) has emerged as an indispensable area of research for future communications due to the global coverage capacity of low Earth orbit (LEO) satellites and the flexible Deployment of aerial platforms. This paper presents a deep reinforcement learning (DRL)-based approach for the joint optimization of offloading and resource allocation in hybrid cloud and multi-access edge computing (MEC) scenarios within SAGINs. The proposed system considers the presence of multiple satellites, clouds and unmanned aerial vehicles (UAVs). The multiple tasks from ground users are modeled as directed acyclic graphs (DAGs). With the goal of reducing energy consumption and latency in MEC, we propose a novel multi-agent algorithm based on DRL that optimizes both the offloading strategy and the allocation of resources in the MEC infrastructure within SAGIN. A hybrid action algorithm is utilized to address the challenge of hybrid continuous and discrete action space in the proposed problems, and a decision-assisted DRL method is adopted to reduce the impact of unavailable actions in the training process of DRL. Through extensive simulations, the results demonstrate the efficacy of the proposed learning-based scheme, the proposed approach consistently outperforms benchmark schemes, highlighting its superior performance and potential for practical applications.