The strength of carrier-sense multiple access with collision avoidance (CSMA/CA) can be combined with that of time-division multiple access (TDMA) to enhance the channel access performance in wireless networks such as the IEEE 802.15.4-based wireless personal area networks (WPANs). In particular, the performance of legacy CSMA/CA-based medium access control (MAC) scheme in congested networks can be enhanced through a hybrid CSMA/CA-TDMA scheme while preserving the scalability property. In this paper, we present distributed and centralized channel access models which follow the transmission strategies based on Markov decision process (MDP) to access both contention period and contention-free period in an intelligent way. The models consider the buffer status as an indication of congestion provided that the offered traffic does not exceed the channel capacity. We extend the models to consider the hidden node collision problem encountered due to the signal attenuation caused by channel fading. The simulation results show that the MDP-based distributed channel access scheme outperforms the legacy slotted CSMA/CA scheme. This scheme also works efficiently in a network consisting of heterogeneous nodes. The centralized model outperforms the distributed model but requires the global information of the network.