Future immersive XR applications will require energy-efficient, high data rate, and low-latency wireless communications in uplink and downlink. One of the key considerations for supporting such XR applications is intelligent and adaptive resource allocation with edge offloading. To address these demands, this paper proposes a reinforcement learning-based resource allocation framework that dynamically allocates uplink and downlink slots while making offloading decisions based on the XR headset's capabilities and network conditions. The paper presents a numerical analysis of the tradeoff between frame loss rate (FLR) and energy efficiency, identifying decision regions for partial offloading to optimize performance. Results show that for the used set of system parameters, partial offloading can extend the coverage area by 55% and reduce energy consumption by up to 34%, compared to always or never offloading. The results demonstrate that the headset's local computing capability plays a crucial role in offloading decisions. Higher computing abilities enable more efficient local processing, reduce the need for offloading, and enhance energy savings.