This paper investigates the design and performance of delayed bit-interleaved coded modulation (DBICM) with low-density parity-check (LDPC) codes. For Gray labeled square $M$-ary quadrature amplitude modulation (QAM) constellations, we investigate the optimal delay scheme with the largest spectrum efficiency of DBICM for a fixed maximum number of delayed time slots and a given signal-to-noise ratio. When analyzing the capacity of DBICM, we find two important properties: the capacity improvement due to delayed coded bits being mapped to the real and imaginary parts of the transmitted symbols are independent of each other; a pair of delay schemes with delayed coded bits having identical bit-channel capacity lead to equivalent DBICM capacity. Using these two properties, we efficiently optimize the delay scheme for any uniform Gray-QAM systems. Furthermore, these two properties enable efficient LDPC code designs regarding unequal error protection via bit-channel type classifications. Moreover, we use protograph-based extrinsic information transfer charts to jointly optimize degree distributions and channel assignments of LDPC codes and propose a constrained progressive edge growth like algorithm to jointly construct LDPC codes and bit-interleavers for DBICM, taking distinctive bit-channel's capacity into account. Simulation results demonstrate that the designed LDPC coded DBICM systems significantly outperform LDPC coded BICM systems.