The co-existence of small cell base stations (SBSs) with conventional macrocell base station is a promising approach to boost the capacity and coverage of cellular networks. However, densifying the network with a viral deployment of SBSs can significantly increase energy consumption. To reduce the reliance on unsustainable energy sources, one can adopt self-powered SBSs that rely solely on energy harvesting. Due to the uncertainty of energy arrival and the finite capacity of energy storage systems, self-powered SBSs must smartly optimize their ON and OFF schedule. In this paper, the problem of ON/OFF scheduling of self-powered SBSs is studied, in the presence of energy harvesting uncertainty with the goal of minimizing the operational costs consisted of energy consumption and transmission delay of a network. For the original problem, we show an algorithm can solve the problem in the illustrative case. To reduce the complexity of the original problem, an approximation is proposed. To solve the approximated problem, a novel approach based on the ski rental framework, a powerful online optimization tool, is proposed. Using this approach, each SBS can effectively decide on its ON/OFF schedule autonomously, without any prior information on future energy arrivals. By using competitive analysis, a deterministic online algorithm (DOA) and a randomized online algorithm (ROA) are developed. ROA is shown to achieve the optimal competitive ratio in the approximation problem. Simulation results show that, compared to a baseline approach, the ROA can yield performance gains reaching up to 15.6% in terms of reduced total energy consumption of SBSs and up to 20.6% in terms of per-SBS network delay reduction. The results shed light on the fundamental aspects that impact the ON time of SBSs while demonstrating that the proposed ROA can reduce up to 69.9% the total cost compared to a baseline approach.