A finite group $G$ is said to be Cayley integral if every undirected Cayley graph $\operatorname{Cay}(G,S)$ on $G$ is integral. In this paper, we introduce three natural extensions of this concept; namely as: Cayley colour integral, $\mathfrak{F}$-Cayley colour integral and normal Cayley integral groups. We characterize the first two families in its entirety. The last family of groups is shown to be coinciding with inverse semi-rational groups introduced by Chillag and Dolfi, thereby providing an alternative characterization for the same. We also establish an inclusion hierarchy among these families.