Data-analysis tasks often involve an iterative process, which requires refining previous solutions. For instance, when analyzing dynamic social networks, we may be interested in monitoring the evolution of a community that was identified at an earlier snapshot. This task requires finding a community in the current snapshot of data that is ``close'' to the earlier-discovered community of interest. However, classic optimization algorithms, which typically find solutions from scratch, potentially return communities that are very dissimilar to the initial one. To mitigate these issues, we introduce the \emph{OptiRefine framework}. The framework optimizes initial solutions by making a small number of \emph{refinements}, thereby ensuring that the new solution remains close to the initial solution and simultaneously achieving a near-optimal solution for the optimization problem. We apply the OptiRefine framework to two classic graph-optimization problems: \emph{densest subgraph} and \emph{maximum cut}. For the \emph{densest-subgraph problem}, we optimize a given subgraph's density by adding or removing $k$~nodes. We show that this novel problem is a generalization of $k$-densest subgraph, and provide constant-factor approximation algorithms for $k=Ω(n)$~refinements. We also study a version of \emph{maximum cut} in which the goal is to improve a given cut. We provide connections to maximum cut with cardinality constraints and provide an optimal approximation algorithm in most parameter regimes under the Unique Games Conjecture for $k=Ω(n)$~refinements. We evaluate our theoretical methods and scalable heuristics on synthetic and real-world data and show that they are highly effective in practice.