We introduce higher-order Stein kernels relative to the standard Gaussian measure, which generalize the usual Stein kernels by involving higher-order derivatives of test functions. We relate the associated discrepancies to various metrics on the space of probability measures and prove new functional inequalities involving them. As an application, we obtain new explicit improved rates of convergence in the classical multidimensional CLT under higher moment and regularity assumptions.