In the rapidly evolving landscape of 5G and beyond, cloud-native Open Radio Access Networks (O-RAN) present a paradigm shift towards intelligent, flexible, and sustainable network operations. This study addresses the intricate challenge of energy efficient (EE) resource allocation that services both enhanced Mobile Broadband (eMBB) and ultra-reliable low-latency communications (URLLC) users. We propose a novel distributed learning framework leveraging on-policy and off-policy transfer learning strategies within a deep reinforcement learning (DRL)--based model to facilitate online resource allocation decisions under different channel conditions. The simulation results explain the efficacy of the proposed method, which rapidly adapts to dynamic network states, thereby achieving a green resource allocation.