This paper studies the variation diminishing property of $k$-positive linear time-invariant (LTI) systems, which map inputs with $k-1$ sign changes to outputs with at most the same variation. We characterize this property for the Toeplitz and Hankel operators of finite-dimensional systems. Our main result is that these operators have a dominant approximation in the form of series or parallel interconnections of $k$ first order positive systems. This is shown by expressing the $k$-positivity of a LTI system as the external positivity (that is, $1$-positivity) of $k$ compound LTI systems. Our characterization generalizes well known properties of externally positive systems ($k=1$) and totally positive systems ($k=\infty$; also known as relaxation systems).