We prove the large deviation principle for several entropy and cross entropy estimators based on return times and waiting times on shift spaces over finite alphabets. We consider shift-invariant probability measures satisfying some decoupling conditions which imply no form of mixing nor ergodicity. We establish precise relations between the rate functions of the different estimators, and between these rate functions and the corresponding pressures, one of which is the Rényi entropy function. For the most commonly used definition of return times, the large-deviation rate function is proved to be nonconvex, except in marginal cases. The results apply in particular to irreducible Markov chains, equilibrium measures for Bowen-regular potentials, g-measures, invariant Gibbs states for absolutely summable interactions in statistical mechanics, and also to probability measures which may be far from Gibbsian, including some hidden Markov models and repeated quantum measurement processes.