Bit-slicing is a software implementation technique that treats an N-bit processor datapath as N parallel single-bit datapaths. The natural spatial redundancy of bit-sliced software can be used to build countermeasures against implementation attacks. While the merits of bit-slicing for side-channel countermeasures have been studied before, their application for protection of post-quantum algorithms against fault injection is still unexplored. We present an end-to-end analysis of the efficacy of bit-slicing to detect and thwart electromagnetic fault injection (EMFI) attacks on post-quantum cryptography (PQC). We study Dilithium, a digital signature finalist of the NIST PQC competition. We present a bit-slice-redundant design for the Number-Theoretic Transform (NTT), the most complex and compute-intensive component in Dilithium. We show a data-redundant countermeasure for NTT which offers two concurrent bits for every single bit in the original implementation. We then implement a full Dilithium signature sequence on a 667 MHz ARM Cortex-A9 processor integrated in a Xilinx Zynq SoC. We perform a detailed EM fault-injection parameter search to optimize the location, intensity and timing of injected EM pulses. We demonstrate that, under optimized fault injection parameters, about 10% of the injected faults become potentially exploitable. However, the bit-sliced NTT design is able to catch the majority of these potentially exploitable faults, even when the remainder of the Dilithium algorithm as well as the control flow is left unprotected. To our knowledge, this is the first demonstration of a bitslice-redundant design of Dilithium that offers distributed fault detection throughout the execution of the algorithm.