For two lacunary sequences $(M_{n,1})_{n\geq 2},(M_{n,2})_{n\geq 0}$ and suitable functions $f$ we introduce random matrix ensembles with \begin{equation*} X_{n,n'}=f(M_{n+n',1}x_1,M_{|n-n'|,2}x_2). \end{equation*} We prove weak convergence of the mean empirical eigenvalue distribution towards the semicircle law under some further number theoretic properties of the sequence $(M_{n,1})_{n\geq 1}$. Furthermore we give examples to show that even in this particular class of random matrix ensembles the asymptotic behaviour of the spectrum becomes delicate. We prove that the empirical spectral distribution does not converge to the semicircle law in general even if the correlation of two entries decays exponentially in the distance. For $f(x_1,x_2)=1/\sqrt{2}\cdot(\cos(2π(x_1+x_2))+\cos(4π(x_1+x_2)))$ and $M_{n,1}=2^n$ we show that the mean empirical spectral distribution does not converge to semicircle law while for any sequence $(M_{n,1})_{n\geq 1}$ with $M_{n+1,1}/M_{n,1}\to\infty$ for $n\to\infty$ and any periodic function $f$ of finite total variation in the sense of Hardy and Krause with mean zero and unit variance the mean spectral distribution converges to the semicircle law.