We study the spectrum of generalized Paley graphs $Γ(k,q)=Cay(\mathbb{F}_q,R_k)$, undirected or not, with $R_k=\{x^k:x\in \mathbb{F}_q^*\}$ where $q=p^m$ with $p$ prime and $k\mid q-1$. We first show that the eigenvalues of $Γ(k,q)$ are given by the Gaussian periods $η_{i}^{(k,q)}$ with $0\le i\le k-1$. Then, we explicitly compute the spectrum of $Γ(k,q)$ with $1\le k \le 4$ and of $Γ(5,q)$ for $p\equiv 1\pmod 5$ and $5\mid m$. Also, we characterize those GP-graphs having integral spectrum, showing that $Γ(k,q)$ is integral if and only if $p$ divides $(q-1)/(p-1)$. Next, we focus on the family of semiprimitive GP-graphs. We show that they are integral strongly regular graphs (of pseudo-Latin square type). Finally, we characterize all integral Ramanujan graphs $Γ(k,q)$ with $1\le k \le 4$ or where $(k,q)$ is a semiprimitive pair.