Paley graphs and Paley sum graphs are classical examples of quasi-random graphs. In this paper, we provide new constructions of families of quasi-random graphs that behave like Paley graphs but are neither Cayley graphs nor Cayley sum graphs. These graphs give a unified perspective of studying various graphs arising from polynomials over finite fields, such as Paley graphs, Paley sum graphs, and graphs arising from Diophantine tuples and their generalizations. We also obtain lower bounds on the clique and independence numbers of the graphs in these families.