The use of multidimensional modulations can decrease the energy consumption of optical links. In this paper, we propose and study Mode Vector Modulation (MVM), a generalized polarization modulation scheme for transmission over multimode/multicore optical fibers or free space. Similar to Polarization Shift Keying (PolSK) and Stokes Vector Modulation (SVM), MVM can be used in conjunction with direct detection and, therefore, is suitable for next-generation, short-haul optical interconnects. This paper focuses on the MVM transceiver architecture, the back-to-back performance of optically-preamplified MVM direct detection (DD) receivers, the optimized geometric shaping of the MVM constellation, and the related bit-to-symbol mapping. We show that MVM DD outperforms conventional single-mode, direct-detection-compliant, digital modulation formats by several dBs in terms of receiver sensitivity and the SNR gain increases with the number of spatial degrees of freedom (SDOFs) N.