This work considers the design of short non-binary low-density parity-check (LDPC) codes over finite fields of order m, for channels with phase noise. In particular, m-ary differential phase-shift keying (DPSK) modulated code symbols are transmitted over an additive white Gaussian noise (AWGN) channel with Wiener phase noise. At the receiver side, non-coherent detection takes place, with the help of a multi-symbol detection algorithm, followed by a non-binary decoding step. Both the detector and decoder operate on a joint factor graph. As a benchmark, finite length bounds and information rate expressions are computed and compared with the codeword error rate (CER) performance, as well as the iterative threshold of the obtained codes. As a result, performance within 1:2 dB from finite-length bounds is obtained, down to a CER of 1e-3.