Orthogonal delay-Doppler (DD) division multiplexing (ODDM) modulation has recently emerged as a promising paradigm for ensuring reliable communications in doubly-selective channels. This work investigates the spectra and orthogonality characteristics of analog (direct) and approximate digital implementations of ODDM systems. We first determine the time and frequency domain representations of the basis functions for waveform in analog and approximate digital ODDM systems. Thereafter, we derive their power spectral densities and show that while the spectrum of analog ODDM waveforms exhibits a step-wise behavior in its transition regions, the spectrum of approximate digital ODDM waveforms is confined to that of the ODDM sub-pulse. Next, we prove the orthogonality characteristics of approximate digital ODDM waveforms and show that, unlike analog ODDM waveforms, the approximate digital ODDM waveforms satisfy orthogonality without the need of additional time domain resources. Additionally, we examine the similarities and differences that implementations of approximate digital ODDM share with the other variants of DD modulations, focusing on the domain changes the symbols undergo, the type of pulse shaping and windowing used, and the domains and the sequence in which they are performed. Finally, we present numerical results to validate our findings and draw further insights.