Zig-Zag is Piecewise Deterministic Markov Process, efficiently used for simulation in an MCMC setting. As we show in this article, it fails to be exponentially ergodic on heavy tailed target distributions. We introduce an extension of the Zig-Zag process by allowing the process to move with a non-constant speed function $s$, depending on the current state of the process. We call this process Speed Up Zig-Zag (SUZZ). We provide conditions that guarantee stability properties for the SUZZ process, including non-explosivity, exponential ergodicity in heavy tailed targets and central limit theorem. Interestingly, we find that using speed functions that induce explosive deterministic dynamics may lead to stable algorithms that can even mix faster. We further discuss the choice of an efficient speed function by providing an efficiency criterion for the one-dimensional process and we support our findings with simulation results.