Distributional properties -including Laplace transforms- of integrals of Markov processes received a lot of attention in the literature. In this paper, we complete existing results in several ways. First, we provide the analytical solution to the most general form of Gaussian processes (with non-stationary increments) solving a stochastic differential equation. We further derive the characteristic function of integrals of Lévy-processes and Lévy driven Ornstein-Uhlenbeck processes with time-inhomogeneous coefficients based on the characteristic exponent of the corresponding stochastic integral. This yields a two-dimensional integral which can be solved explicitly in a lot of cases. This applies to integrals of compound Poisson processes, whose characteristic function can then be obtained in a much easier way than using joint conditioning on jump times. Closed form expressions are given for gamma-distributed jump sizes as an example.