For stochastic diffusion processes the dominant eigenfunctions of the corresponding Koopman operator contain important information about the slow-scale dynamics, that is, about the location and frequency of rare events. In this article, we reformulate the eigenproblem in terms of $χ$-functions in the ISOKANN framework and discuss how optimal control and importance sampling allows for zero variance sampling of these functions. We provide a new formulation of the ISOKANN algorithm allowing for a proof of convergence and incorporate the optimal control result to obtain an adaptive iterative algorithm alternating between importance sampling and $χ$-function approximation. We demonstrate the usage of our proposed method in experiments increasing the approximation accuracy by several orders of magnitude.