Transmission of a Gaussian source over a time-varying multiple-input multiple-output (MIMO) channel is studied under strict delay constraints. Availability of a correlated side information at the receiver is assumed, whose quality, i.e., correlation with the source signal, also varies over time. A block-fading model is considered for the states of the time-varying channel and the time-varying side information; and perfect state information at the receiver is assumed, while the transmitter knows only the statistics. The high SNR performance, characterized by the \textit{distortion exponent}, is studied for this joint source-channel coding problem. An upper bound is derived and compared with lowers based on list decoding, hybrid digital-analog transmission, as well as multi-layer schemes which transmit successive refinements of the source, relying on progressive and superposed transmission with list decoding. The optimal distortion exponent is characterized for the single-input multiple-output (SIMO) and multiple-input single-output (MISO) scenarios by showing that the distortion exponent achieved by multi-layer superpositon encoding with joint decoding meets the proposed upper bound. In the MIMO scenario, the optimal distortion exponent is characterized in the low bandwidth ratio regime, and it is shown that the multi-layer superposition encoding performs very close to the upper bound in the high bandwidth expansion regime.