The question of existence and properties of stationary solutions to Langevin equations driven by noise processes with stationary increments is discussed, with particular focus on noise processes of pseudo-moving-average type. On account of the Wold-Karhunen decomposition theorem, such solutions are, in principle, representable as a moving average (plus a drift-like term) but the kernel in the moving average is generally not available in explicit form. A class of cases is determined where an explicit expression of the kernel can be given, and this is used to obtain information on the asymptotic behavior of the associated autocorrelation functions, both for small and large lags. Applications to Gaussian- and Lévy-driven fractional Ornstein-Uhlenbeck processes are presented. A Fubini theorem for Lévy bases is established as an element in the derivations.