We provide an extensive study of the differential properties of the functions $x\mapsto x^{2^t-1}$ over $\F$, for $2 \leq t \leq n-1$. We notably show that the differential spectra of these functions are determined by the number of roots of the linear polynomials $x^{2^t}+bx^2+(b+1)x$ where $b$ varies in $\F$.We prove a strong relationship between the differential spectra of $x\mapsto x^{2^t-1}$ and $x\mapsto x^{2^{s}-1}$ for $s= n-t+1$. As a direct consequence, this result enlightens a connection between the differential properties of the cube function and of the inverse function. We also determine the complete differential spectra of $x \mapsto x^7$ by means of the value of some Kloosterman sums, and of $x \mapsto x^{2^t-1}$ for $t \in \{\lfloor n/2\rfloor, \lceil n/2\rceil+1, n-2\}$.