In this paper we propose a rapidly converging LMS algorithm for crosstalk cancellation. The architecture is similar to deep neural networks, where multiple layers are adapted sequentially. The application motivating this approach is gigabit rate transmission over unshielded twisted pairs using a vectored system. The crosstalk cancellation algorithm uses an adaptive non-diagonal preprocessing matrix prior to a conventional LMS crosstalk canceler. The update of the preprocessing matrix is inspired by deep neural networks. However, since most the operations in the Deep-LMS algorithm are linear, we are capable of providing an exact convergence speed analysis. The role of the preprocessing matrix is to speed up the convergence of the conventional LMS crosstalk canceler and hence the convergence of the overall system. The Deep-LMS is important for crosstalk cancellation in the novel G.fast standard, where traditional LMS converges very slowly due to the ill-conditioned covariance matrix of the received signal at the extended bandwidth. Simulation results support our analysis and show significant reduction in convergence time compared to existing LMS variants.