In this paper, we mainly study the Castelnuovo-Mumford regularity of the generalized binomial edge ideals of graphs. We show that this number can be any integer number from $2$ to $n-1$ where $n$ is the number of vertices in the underlying graph. We are able to show this, after giving some tight lower and upper bounds for the regularity of generalized binomial edge ideals of the join product of graphs. In particular, we characterize all generalized binomial edge ideals with the regularity equal to~$2$ as well as extremal Gorenstein ideals. For this purpose, we give a new combinatorial characterization for the class of $P_4$-free graphs.