We study the peak height distribution of certain non-stationary Gaussian random fields. The explicit peak height distribution of smooth, non-stationary Gaussian processes in 1D with general covariance is derived. The formula is determined by two parameters, each of which has a clear statistical meaning. For multidimensional non-stationary Gaussian random fields, we generalize these results to the setting of scale space fields, which play an important role in peak detection by helping to handle peaks of different spatial extents. We demonstrate that these properties not only offer a better interpretation of the scale space field but also simplify the computation of the peak height distribution. Finally, two efficient numerical algorithms are proposed as a general solution for computing the peak height distribution of smooth multidimensional Gaussian random fields in applications.