Orthogonal Time Frequency Space (OTFS) is a novel modulation scheme designed in the Doppler-delay domain to fully exploit time and frequency diversity of general time-varying channels. In this paper, we present a novel discrete-time analysis of OFDM-based OTFS transceiver with a concise and vectorized input-output relationship that clearly characterizes the contribution of each underlying signal processing block in such systems. When adopting cyclic prefix in the time domain, our analysis reveals that the proposed MIMO OTFS and OFDM systems have the same ergodic capacity despite the well-known fact that the former has great advantages in low-complexity receiver design for high Doppler channels. The proposed discrete-time vectorized formulation is applicable to general fast fading channels with arbitrary window functions. It also enables practical low-complexity receiver design for which such a concise formulation of the input-output relationship is of great benefits.