This paper studies the fundamental problem of graph coloring in fully dynamic graphs. Since the problem of computing an optimal coloring, or even approximating it to within $n^{1-ε}$ for any $ε> 0$, is NP-hard in static graphs, there is no hope to achieve any meaningful computational results for general graphs in the dynamic setting. It is therefore only natural to consider the combinatorial aspects of dynamic coloring, or alternatively, study restricted families of graphs. Towards understanding the combinatorial aspects of this problem, one may assume a black-box access to a static algorithm for $C$-coloring any subgraph of the dynamic graph, and investigate the trade-off between the number of colors and the number of recolorings per update step. In WADS'17, Barba et al. devised two complementary algorithms: For any $β> 0$ the first (respectively, second) maintains an $O(C βn^{1/β})$ (resp., $O(C β)$)-coloring while recoloring $O(β)$ (resp., $O(βn^{1/β})$) vertices per update. Our contribution is two-fold: - We devise a new algorithm for general graphs that improves significantly upon the first trade-off in a wide range of parameters: For any $β> 0$, we get a $\tilde{O}(\frac{C}β\log^2 n)$-coloring with $O(β)$ recolorings per update, where the $\tilde{O}$ notation supresses polyloglog$(n)$ factors. In particular, for $β=O(1)$ we get constant recolorings with polylog$(n)$ colors; this is an exponential improvement over the previous bound. - For uniformly sparse graphs, we use low out-degree orientations to strengthen the above result by bounding the update time of the algorithm rather than the number of recolorings. Then, we further improve this result by introducing a new data structure that refines bounded out-degree edge orientations and is of independent interest.