As an enhanced version of massive machine-type communication in 5G, massive communication has emerged as one of the six usage scenarios anticipated for 6G, owing to its potential in industrial internet-of-things and smart metering. Driven by the need for random multiple-access (RMA) in massive communication, as well as, next-generation Wi-Fi, medium access control has attracted considerable recent attention. Holding the promise of attaining bandwidth-efficient collision resolution, multiaccess reservation no doubt plays a central role in RMA, e.g., the distributed coordination function (DCF) in IEEE 802.11. In this paper, we are interested in maximizing the bandwidth efficiency of reservation protocols for RMA under quality-of-service constraints. Particularly, we present a tree splitting based reservation scheme, in which the attempting probability is dynamically optimized by partially observable Markov decision process or reinforcement learning (RL). The RL-empowered tree-splitting algorithm guarantees that all these terminals with backlogged packets at the beginning of a contention cycle can be scheduled, thereby providing a first-in-first-out service. More importantly, it substantially reduces the reservation bandwidth determined by the communication complexity of DCF, through judiciously conceived coding and interaction for exchanging information required by distributed ordering. Simulations demonstrate that the proposed algorithm outperforms the CSMA/CA based DCF in IEEE 802.11.