The problem of finding decentralized transmission policies in a wireless communication network with energy harvesting constraints is formulated and solved using the decentralized Markov decision process framework. The proposed policy defines the transmission probabilities of all devices so as to correctly balance the collision probabilities with the energy constraints. After an initial coordination phase, in which the network parameters are initialized for all devices, every node proceeds in a fully decentralized fashion. We numerically show that, because of the harvesting, a fully orthogonal scheme (e.g., TDMA-like) is sub-optimal in this scenario, and that the optimal trade-off lies between an orthogonal and a completely symmetric system.