It is well-known that, on a purely algebraic level, a simplified algebraic version of the Central Limit Theorem (CLT) can be proved in the framework of a noncommutative probability space, under the hypotheses that the sequence of non-commutative random variables we consider is exchangeable and obeys a certain vanishing condition of some of its joint moments. In this approach (which covers versions for both the classical CLT and the CLT of free probability), the determination of the resulting limit law has to be addressed on a case-by-case basis. In this paper we discuss an instance of the above theorem which takes place in the framework of the group algebra of the infinite symmetric group $S_{\infty}$: the exchangeable sequence that is considered consists of the star-generators of $S_{\infty}$, and the expectation functional used on the group algebra of $S_{\infty}$ depends in a natural way on a parameter $d$, which is a positive integer. We identify precisely the limit distribution $μ_d$ for this special instance of exchangeable CLT, via a connection that $μ_d$ turns out to have with the average empirical eigenvalue distribution of a random GUE matrix of size $d \times d$. Moreover, we put into evidence a multi-variate version of this result which follows from the observation that, on the level of calculations with pair-partitions, the (non-centred) star-generators are related to a (centred) exchangeable sequence of GUE matrices with independent entries