Motivated by entropic optimal transport, time reversal of Markov jump processes in $\mathbb{R}^n$ is investigated. Relying on an abstract integration by parts formula for the carré du champ of a Markov process recently obtained by Cattiaux, Gentil and the authors, and using an entropic improvement strategy discovered by Föllmer in the eighties, we compute the semimartingale characteristics of the time reversed process for a wide class of jump processes with possibly unbounded variation sample paths and singular intensities of jump.