Polar codes are the first provable capacity-achieving forward error correction (FEC) codes. In general polar codes can be decoded via either successive cancellation (SC) or belief propagation (BP) decoding algorithm. However, to date practical applications of polar codes have been hindered by the long decoding latency and limited error-correcting performance problems. In this paper, based on our recent proposed early stopping criteria for the BP algorithm, we propose a hybrid BP-SC decoding scheme to improve the decoding performance of polar codes with relatively short latency. Simulation results show that, for (1024, 512) polar codes the proposed approach leads to at least 0.2dB gain over the BP algorithm with the same maximum number of iterations for the entire SNR region, and also achieves 0.2dB decoding gain over the BP algorithm with the same worst-case latency in the high SNR region. Besides, compared to the SC algorithm, the proposed scheme leads to 0.2dB gain in the medium SNR region with much less average decoding latency. In addition, we also propose the low-complexity unified hardware architecture for the hybrid decoding scheme, which is able to implement SC and BP algorithms using same hardware.