We introduce a novel numerical approach for a class of stochastic dynamic programs which arise as discretizations of backward stochastic differential equations or semi-linear partial differential equations. Solving such dynamic programs numerically requires the approximation of nested conditional expectations, i.e., iterated integrals of previous approximations. Our approach allows us to compute and iteratively improve upper and lower bounds on the true solution starting from an arbitrary and possibly crude input approximation. We demonstrate the benefits of our approach in a high dimensional financial application.