We consider a generic one-dimensional stochastic process $x(t)$, or a random walk $X_n$, which describes the position of a particle evolving inside an interval $[a,b]$, with absorbing walls located at $a$ and $b$. In continuous time, $x(t)$ is driven by some equilibrium process $\mathbfθ(t)$, while in discrete time, the jumps of $X_n$ follow a stationary process that obeys a time reversal property. An important observable to characterize its behaviour is the exit probability $E_b(x,t)$, which is the probability for the particle to be absorbed first at the wall $b$, before or at time $t$, given its initial position $x$. In this paper we show that the derivation of this quantity can be tackled by studying a dual process $y(t)$ very similar to $x(t)$ but with hard walls at $a$ and $b$. More precisely, we show that the quantity $E_b(x,t)$ for the process $x(t)$ is equal to the probability $\tilde Φ(x,t|b)$ of finding the dual process inside the interval $[a,x]$ at time $t$, with $y(0) =b$. This is known as Siegmund duality in mathematics. Here we show that this duality applies to various processes which are of interest in physics, including models of active particles, diffusing diffusivity models, a large class of discrete and continuous time random walks, and even processes subjected to stochastic resetting. For all these cases, we provide an explicit construction of the dual process. We also give simple derivations of this identity both in the continuous and in the discrete time setting, as well as numerical tests for a large number of models of interest. Finally, we use simulations to show that the duality is also likely to hold for more complex processes such as fractional Brownian motion.