We present a tutorial on reduced-rank signal processing, design methods and algorithms for dimensionality reduction, and cover a number of important applications. A general framework based on linear algebra and linear estimation is employed to introduce the reader to the fundamentals of reduced-rank signal processing and to describe how dimensionality reduction is performed on an observed discrete-time signal. A unified treatment of dimensionality reduction algorithms is presented with the aid of least squares optimization techniques, in which several techniques for designing the transformation matrix that performs dimensionality reduction are reviewed. Among the dimensionality reduction techniques are those based on the eigen-decomposition of the observed data vector covariance matrix, Krylov subspace methods, joint and iterative optimization (JIO) algorithms and JIO with simplified structures and switching (JIOS) techniques. A number of applications are then considered using a unified treatment, which includes wireless communications, sensor and array signal processing, and speech, audio, image and video processing. This tutorial concludes with a discussion of future research directions and emerging topics.