The partial coloring method is one of the most powerful and widely used method in combinatorial discrepancy problems. However, in many cases it leads to sub-optimal bounds as the partial coloring step must be iterated a logarithmic number of times, and the errors can add up in an adversarial way. We give a new and general algorithmic framework that overcomes the limitations of the partial coloring method and can be applied in a black-box manner to various problems. Using this framework, we give new improved bounds and algorithms for several classic problems in discrepancy. In particular, for Tusnady's problem, we give an improved $O(\log^2 n)$ bound for discrepancy of axis-parallel rectangles and more generally an $O_d(\log^dn)$ bound for $d$-dimensional boxes in $\mathbb{R}^d$. Previously, even non-constructively, the best bounds were $O(\log^{2.5} n)$ and $O_d(\log^{d+0.5}n)$ respectively. Similarly, for the Steinitz problem we give the first algorithm that matches the best known non-constructive bounds due to Banaszczyk [Banaszczyk 2012] in the $\ell_\infty$ case, and improves the previous algorithmic bounds substantially in the $\ell_2$ case. Our framework is based upon a substantial generalization of the techniques developed recently in the context of the Komlós discrepancy problem [BDG16].