Finite fields of the form GF(2^m) play an important role in coding theory and cryptography. We show that the choice of how to represent the elements of these fields can have a significant impact on the resource requirements for quantum arithmetic. In particular, we show how the use of Gaussian normal basis representations and of `ghost-bit basis' representations can be used to implement inverters with a quantum circuit of depth O(m log(m)). To the best of our knowledge, this is the first construction with subquadratic depth reported in the literature. Our quantum circuit for the computation of multiplicative inverses is based on the Itoh-Tsujii algorithm which exploits that in normal basis representation squaring corresponds to a permutation of the coefficients. We give resource estimates for the resulting quantum circuit for inversion over binary fields GF(2^m) based on an elementary gate set that is useful for fault-tolerant implementation.