To achieve high data rates and better connectivity in future communication networks, the deployment of different types of access points (APs) is underway. In order to limit human intervention and reduce costs, the APs are expected to be equipped with self-organizing capabilities. Moreover, due to the spectrum crunch, frequency reuse among the deployed APs is inevitable, aggravating the problem of inter-cell interference (ICI). Therefore, ICI mitigation in self-organizing networks (SONs) is commonly identified as a key radio resource management mechanism to enhance performance in future communication networks. With the aim of reducing ICI in a SON, this paper proposes a novel solution for the uncoordinated channel and power allocation problems. Based on the multi-player multi-armed bandit (MAB) framework, the proposed technique does not require any communication or coordination between the APs. The case of varying channel rewards across APs is considered. In contrast to previous work on channel allocation using the MAB framework, APs are permitted to choose multiple channels for transmission. Moreover, non-orthogonal multiple access (NOMA) is used to allow multiple APs to access each channel simultaneously. This results in an MAB model with varying channel rewards, multiple plays and non-zero reward on collision. The proposed algorithm has an expected regret in the order of O(log^2 T ), which is validated by simulation results. Extensive numerical results also reveal that the proposed technique significantly outperforms the well-known upper confidence bound (UCB) algorithm, by achieving more than a twofold increase in the energy efficiency.