Efficient decoding is crucial to high-throughput and power-sensitive wireless communication scenarios. A theoretical analysis of the performance-complexity tradeoff toward low-complexity decoding is required for a better understanding of the fundamental limits in the above-mentioned scenarios. This study aims to explore the performance of LDPC codes under belief propagation (BP) decoding with complexity constraints. In other words, for a small number of iterations, we present a closed-form lower bound on the bit error rate (BER) of LDPC codes as a function of complexity. Specifically, for the regular LDPC code ensembles, the dominant term in the order of the lower bound we provide matches that of the upper bound given by Lentmaier. Furthermore, for irregular LDPC code ensembles, in addition to adopting the approach used for regular codes, we also propose a method to iteratively obtain the lower bound on the average BER.