Aldous' spectral gap conjecture, proven by Caputo, Liggett and Richthammer, states the following: for any set of transpositions in the symmetric group $\mathrm{Sym}(n)$, the spectral gap of the corresponding random walk on the group -- an $n!$-state process -- coincides with that of the corresponding random walk of a single element -- an $n$-state process. This paper presents an analog of this conjecture in the unitary group $\mathrm{U}(n)$, and proves it in several non-trivial cases. The phenomenon we discover is that for some natural families of probability distributions on $\mathrm{U}(n)$, the spectral gap of the corresponding random walk, which has a continuous state space, is identical to that of a discrete KMP process (also known as the uniform reshuffling process) with two indistinguishable particles on a hypergraph on $n$ vertices -- a discrete Markov chain with $\binom{n+1}{2}$ states.