The Cayley graphs of finite groups are known to provide several examples of families of expanders, and some of them are Ramanujan graphs. Babai studied isospectral non-isomorphic Cayley graphs of the dihedral groups. Lubotzky, Samuels and Vishne proved that there are isospectral non-isomorphic Cayley graphs of $\mathrm{PSL}_d(\mathbb F_q)$ for every $d\geq 5$ ($d \neq 6$) and prime power $q> 2$. In this article, we focus on three variants of Cayley graphs, viz., the Cayley sum graphs, the twisted Cayley graphs, and the twisted Cayley sum graphs. We prove the existence of non-isomorphic expander families of bounded degree, whose spectra are related by the values of certain characters. We also provide several new examples of expander families, and examples of non-expanders and Ramanujan graphs formed by these three variants.