Cell-free massive multi-input multi-output (MIMO) promises uniform high performance across the network, but also brings a high energy cost due to joint transmission from distributed radio units (RUs) and centralized processing in the cloud. Leveraging the resource-sharing capabilities of Open Radio Access Network (O-RAN), we propose EARL, an energy-aware adaptive antenna control framework based on reinforcement learning. EARL dynamically configures antenna elements in RUs to minimize radio, optical fronthaul, and cloud processing power consumption while meeting user spectral efficiency demands. Numerical results show power savings of up to 81% and 50% over full-on and heuristic baselines, respectively. The RL-based approach operates within 220 ms, satisfying O-RAN's near-real-time limit, and a greedy refinement further halves power consumption at a 2 s runtime.