We consider directed polymers in random environment on the lattice Z d at small inverse temperature and dimension d $\ge$ 3. Then, the normalized partition function W n is a regular martingale with limit W. We prove that n (d--2)/4 (W n -- W)/W n converges in distribution to a Gaussian law. Both the polynomial rate of convergence and the scaling with the martingale W n are different from those for polymers on trees.