This paper focuses on the study of an original combination of the Multilevel Monte Carlo method introduced by Giles [10] and the popular importance sampling technique. To compute the optimal choice of the parameter involved in the importance sampling method, we rely on Robbins-Monro type stochastic algorithms. On the one hand, we extend our previous work [2] to the Multilevel Monte Carlo setting. On the other hand, we improve [2] by providing a new adaptive algorithm avoiding the discretization of any additional process. Furthermore, from a technical point of view, the use of the same stochastic algorithms as in [2] appears to be problematic. To overcome this issue, we employ an alternative version of stochastic algorithms with projection (see e.g. Laruelle, Lehalle and Pagès [20]). In this setting, we show innovative limit theorems for a doubly indexed stochastic algorithm which appear to be crucial to study the asymptotic behavior of the new adaptive Multilevel Monte Carlo estimator. Finally, we illustrate the efficiency of our method through applications from quantitative finance.