Low density parity check (LDPC) codes, initially discovered by Gallager, exhibit excellent performance in iterative decoding, approaching the Shannon limit. MDS array codes, with favorable algebraic structures, are codes suitable for decoding large burst errors. The Blaum-Roth (BR) code, an MDS array code similar to the Reed-Solomon (RS) code but has a parity-check matrix prone to $4$-cycles. Fossorier proposed constructing quasi-cyclic LDPC codes from circulant permutation matrices but are not MDS array codes. This paper aims to construct codes that possess both the block MDS property and have no $4$-cycles in the Tanner graph of their parity-check matrices, namely the so-called block MDS LDPC codes. Non-binary block MDS QC codes were first constructed by [Tauz {\it et al. }IEEE ITW, 2025] using circulant shift matrices. We first generate a family of block MDS codes over $\F_2$ from punctured circulant permutation matrices. Second, we construct a family of block MDS LDPC codes from circulant matrices with column weight $> 1$ (CM$(t)$). Additionally, we present the Moore determinant formula for CM$(t)$s and a sufficient condition to avoid $4$-cycles in CM\((t)\)-QC LDPC codes' Tanner graphs for $t> 1$. We also point out the non-existence of binary block MDS CPM-QC LDPC codes. Compared to the codes constructed in [Li {\it et al. }IEEE TIT, 2023] and [Xiao {\it et al. }IEEE TCOM, 2021], our block MDS LDPC codes show enhanced random-error-correction at a similar code length and rate. Meanwhile, these codes can effectively combat burst errors when considered as array codes. Both of our two types of constructions for block MDS LDPC codes are applicable to the scenario of the binary field.