As intelligent reflecting surface (IRS) has emerged as a new and promising technology capable of configuring the wireless environment favorably, channel estimation for IRS-assisted multiple-input multiple-output (MIMO) systems has garnered extensive attention in recent years. Despite the development of numerous algorithms to address this challenge, a comprehensive theoretical characterization of the optimal recovery error is still lacking. This paper aims to address this gap by providing theoretical guarantees in terms of stable recovery of channel matrices for noisy measurements. We begin by establishing the equivalence between IRS-assisted MIMO systems in the uplink scenario and a compact tensor train (TT)-based tensor-on-tensor (ToT) regression. Building on this equivalence, we then investigate the restricted isometry property (RIP) for complex-valued subgaussian measurements. Our analysis reveals that successful recovery hinges on the relationship between the number of user terminals and the number of time slots during which channel matrices remain invariant. Utilizing the RIP condition, we establish a theoretical upper bound on the recovery error for solutions to the constrained least-squares optimization problem, as well as a minimax lower bound for the considered model. Our analysis demonstrates that the recovery error decreases inversely with the number of time slots, and increases proportionally with the total number of unknown entries in the channel matrices, thereby quantifying the fundamental trade-offs in channel estimation accuracy. In addition, we explore a multi-hop IRS scheme and analyze the corresponding recovery errors. Finally, we have performed numerical experiments to support our theoretical findings.