We discuss how spreading processes on temporal networks are impacted by the shape of their inter-event time distributions. Through simple mathematical arguments and toy examples, we find that the key factor is the ordering in which events take place, a property that tends to be affected by the bulk of the distributions and not only by their tail, as usually considered in the literature. We show that a detailed modeling of the temporal patterns observed in complex networks can change dramatically the properties of a spreading process, such as the ergodicity of a random walk process or the persistence of an epidemic.