In the conventional successive cancellation (SC) decoder for polar codes, all the future bits to be estimated later are treated as random variables. However, polar codes inevitably involve frozen bits, and their concatenated coding schemes also include parity bits (or dynamic frozen bits) causally generated from the past bits estimated earlier. We refer to the frozen and parity bits located behind a target decoding bit as its \textit{future constraints (FCs)}. Although the values of FCs are deterministic given the past estimates, they have not been exploited in the conventional SC-based decoders, not leading to optimality. In this paper, with a primary focus on the binary erasure channel (BEC), we propose SC-check (SCC) and belief propagation SCC (BP-SCC) decoding algorithms in order to leverage FCs in decoding. We further devise an improved tree search technique based on stack-based backjumping (SBJ) to solve dynamic constraint satisfaction problems (CSPs) formulated by FCs. Over the BEC, numerical results show that a combination of the BP-SCC algorithm and the SBJ tree search technique achieves the erasure recovery performance close to the dependence testing (DT) bound, a bound of achievable finite-length performance.