Kernel-based random graphs (KBRGs) are a broad class of random graph models that account for inhomogeneity among vertices. We consider KBRGs on a discrete $d-$dimensional torus $\mathbf{V}_N$ of size $N^d$. Conditionally on an i.i.d.~sequence of {Pareto} weights $(W_i)_{i\in \mathbf{V}_N}$ with tail exponent $τ-1>0$, we connect any two points $i$ and $j$ on the torus with probability $$p_{ij}= \frac{κ_σ(W_i,W_j)}{\|i-j\|^α} \wedge 1$$ for some parameter $α>0$ and $κ_σ(u,v)= (u\vee v)(u \wedge v)^σ$ for some $σ\in(0,τ-1)$. We focus on the adjacency operator of this random graph and study its empirical spectral distribution. For $α<d$ and $τ>2$, we show that a non-trivial limiting distribution exists as $N\to\infty$ and that the corresponding measure $μ_{σ,τ}$ is absolutely continuous with respect to the Lebesgue measure. $μ_{σ,τ}$ is given by an operator-valued semicircle law, whose Stieltjes transform is characterised by a fixed point equation in an appropriate Banach space. We analyse the moments of $μ_{σ,τ}$ and prove that the second moment is finite even when the weights have infinite variance. In the case $σ=1$, corresponding to the so-called scale-free percolation random graph, we can explicitly describe the limiting measure and study its tail.