We study coherence distillation under time-translation-invariant operations: given many copies of a quantum state containing coherence in the energy eigenbasis, the aim is to produce a purer coherent state while respecting the time-translation symmetry. This symmetry ensures that the output remains synchronized with the input and that the process can be realized by energy-conserving unitaries coupling the system to a reservoir initially in an energy eigenstate, thereby modeling thermal operations supplemented by a work reservoir or battery. For qubit systems, we determine the optimal asymptotic fidelity and show that it is governed by the purity of coherence, a measure of asymmetry derived from the right logarithmic derivative (RLD) Fisher information. In particular, we find that the lowest achievable infidelity (one minus fidelity) scales as $1/N$ times the reciprocal of the purity of coherence of each input qubit, where $N$ is the number of copies, giving this quantity a clear operational meaning. We additionally study many other interesting aspects of the coherence distillation problem for qubits, including computing higher-order corrections to the lowest achievable infidelity up to $O(1/N^3)$, and expressing the optimal channel as a boundary value problem that can be solved numerically.