Linear detectors such as zero forcing (ZF) or minimum mean square error (MMSE) are imperative for large/massive MIMO systems for both the downlink and uplink scenarios. However these linear detectors require matrix inversion which is computationally expensive for such huge systems. In this paper, we assert that calculating an exact inverse is not necessary to find the ZF/MMSE solution and an approximate inverse would yield a similar performance. This is possible if the quantized solution calculated using the approximate inverse is same as the one calculated using the exact inverse. We quantify the amount of approximation that can be tolerated for this to happen. Motivated by this, we propose to use the existing iterative methods for obtaining low complexity approximate inverses. We show that, after a sufficient number of iterations, the inverse using iterative methods can provide a similar error performance. In addition, we also show that the advantage of using an approximate inverse is not limited to linear detectors but can be extended to non linear detectors such as sphere decoders (SD). An approximate inverse can be used for any SD that requires matrix inversion. We prove that application of approximate inverse leads to a smaller radius, which in turn reduces the search space leading to reduction in complexity. Numerical results corroborate our claim that using approximate matrix inversion reduces decoding complexity in large/massive MIMO systems with no loss in error performance.