In this note we show that the singular probability of the adjacency matrix of a random $d$-regular graph on $n$ vertices, where $d$ is fixed and $n \to \infty$, is bounded by $n^{-1/3+o(1)}$. This improves a recent bound by Huang. Our method is based on the study of the singularity problem modulo a prime together with an inverse-type result on the decay of the characteristic function. The latter is related to the inverse Kneser's problem in combinatorics.