This short review describes mathematical techniques for statistical analysis and prediction in dynamical systems. Two problems are discussed, namely (i) the supervised learning problem of forecasting the time evolution of an observable under potentially incomplete observations at forecast initialization; and (ii) the unsupervised learning problem of identification of observables of the system with a coherent dynamical evolution. We discuss how ideas from from operator-theoretic ergodic theory combined with statistical learning theory provide an effective route to address these problems, leading to methods well-adapted to handle nonlinear dynamics, with convergence guarantees as the amount of training data increases.