Complex unit gain graphs may exhibit various kinds of symmetry. In this work, we explore structural symmetry, spectral symmetry and sign-symmetry in such graphs, and their respective relations to one-another. Our main result is a construction that transforms an arbitrary complex unit gain graph into infinitely many switching-distinct ones whose spectral symmetry does not imply sign-symmetry. This provides a more general answer to the analogue of an existence question that was recently treated in the context of signed graphs.