Given $d\ge 1$, we provide a construction of the random measure - the critical Gaussian Multiplicative Chaos - formally defined $e^{\sqrt{2d}X}\mathrm{d} μ$ where $X$ is a $\log$-correlated Gaussian field and $μ$ is a locally finite measure on $\mathbb R^d$. Our construction generalizes the one performed in the case where $μ$ is the Lebesgue measure. It requires that the measure $μ$ is sufficiently spread out, namely that for $μ$ almost every $x$ we have $$ \int_{B(0,1)}\frac{μ(\mathrm{d} y)}{|x-y|^{d}e^{ρ\left(\log \frac{1}{|x-y|} \right)}}<\infty, $$ for any compact set where $ρ:\mathbb R_+\to \mathbb R_+$ can be chosen to be any lower envelope function for the $3$-Bessel process (this includes $ρ(x)=x^α$ with $α\in (0,1/2)$). We prove that three distinct random objects converge to a common limit which defines the critical GMC: the derivative martingale, the critical martingale, and the exponential of the mollified field. We also show that the above criterion for the measure $μ$ is in a sense optimal.