We consider a one-dimensional Brownian motion of fixed duration $T$. Using a path-integral technique, we compute exactly the probability distribution of the difference $τ=t_{\min}-t_{\max}$ between the time $t_{\min}$ of the global minimum and the time $t_{\max}$ of the global maximum. We extend this result to a Brownian bridge, i.e. a periodic Brownian motion of period $T$. In both cases, we compute analytically the first few moments of $τ$, as well as the covariance of $t_{\max}$ and $t_{\min}$, showing that these times are anti-correlated. We demonstrate that the distribution of $τ$ for Brownian motion is valid for discrete-time random walks with $n$ steps and with a finite jump variance, in the limit $n\to \infty$. In the case of Lévy flights, which have a divergent jump variance, we numerically verify that the distribution of $τ$ differs from the Brownian case. For random walks with continuous and symmetric jumps we numerically verify that the probability of the event "$τ= n$" is exactly $1/(2n)$ for any finite $n$, independently of the jump distribution. Our results can be also applied to describe the distance between the maximal and minimal height of $(1+1)$-dimensional stationary-state Kardar-Parisi-Zhang interfaces growing over a substrate of finite size $L$. Our findings are confirmed by numerical simulations. Some of these results have been announced in a recent Letter [Phys. Rev. Lett. 123, 200201 (2019)].