In this article, we explore the spectral properties of general random kernel matrices $[K(U_i,U_j)]_{1\leq i\neq j\leq n}$ from a Lipschitz kernel $K$ with $n$ independent random variables $U_1,U_2,\ldots, U_n$ distributed uniformly over $[0,1]$. In particular we identify a dichotomy in the extreme eigenvalue of the kernel matrix, where, if the kernel $K$ is degenerate, the largest eigenvalue of the kernel matrix (after proper normalization) converges weakly to a weighted sum of independent chi-squared random variables. In contrast, for non-degenerate kernels, it converges to a normal distribution extending and reinforcing earlier results from Koltchinskii and Giné (2000). Further, we apply this result to show a dichotomy in the asymptotic behavior of extreme eigenvalues of $W$-random graphs, which are pivotal in modeling complex networks and analyzing large-scale graph behavior. These graphs are generated using a kernel $W$, termed as graphon, by connecting vertices $i$ and $j$ with probability $W(U_i, U_j)$. Our results show that for a Lipschitz graphon $W$, if the degree function is constant, the fluctuation of the largest eigenvalue (after proper normalization) converges to the weighted sum of independent chi-squared random variables and an independent normal distribution. Otherwise, it converges to a normal distribution.