In this paper, all graphs whose adjacency matrix has at most two eigenvalues (multiplicities included) different from $2$ and $-1$ are determined. These graphs conclude a class of generalized friendship graphs $F_{t,r,k}, $ which is the graph of $k$ copies of the complete graph $K_t$ meeting in common $r$ vertices such that $t-r=3.$ Which of these graphs are determined by its spectrum is are also obtained.