In this paper, we propose a reduced-complexity optimal modified sphere decoding (MSD) detection scheme for SCMA. As SCMA systems are characterized by a number of resource elements (REs) that are less than the number of the supported users, the channel matrix is rank-deficient, and sphere decoding (SD) cannot be directly applied. Inspired by the Tikhonov regularization, we formulate a new full-rank detection problem that it is equivalent to the original rank-deficient detection problem for constellation points with constant modulus and an important subset of non-constant modulus constellations. By exploiting the SCMA structure, the computational complexity of MSD is reduced compared with the conventional SD. We also employ list MSD to facilitate channel coding. Simulation results demonstrate that in uncoded SCMA systems the proposed MSD achieves the performance of the optimal maximum likelihood (ML) detection. Additionally, the proposed MSD benefits from a lower average complexity compared with MPA.