We show that for an $n\times n$ random matrix $A$ with independent uniformly anti-concentrated entries, such that $\mathbb{E} ||A||^2_{HS}\leq K n^2$, the smallest singular value $σ_n(A)$ of $A$ satisfies $$ P\left( σ_n(A)\leq \frac{\varepsilon}{\sqrt{n}} \right) \leq C\varepsilon+2e^{-cn},\quad \varepsilon \ge 0. $$ This extends earlier results of Rudelson and Vershynin, and Rebrova and Tikhomirov by removing the assumption of mean zero and identical distribution of the entries across the matrix, as well as the recent result of Livshyts, where the matrix was required to have i.i.d. rows. Our model covers "inhomogeneus" matrices allowing different variances of the entries, as long as the sum of the second moments is of order $O(n^2)$. In the past advances, the assumption of i.i.d. rows was required due to lack of Littlewood--Offord--type inequalities for weighted sums of non-i.i.d. random variables. Here, we overcome this problem by introducing the Randomized Least Common Denominator (RLCD) which allows to study anti-concentration properties of weighted sums of independent but not identically distributed variables. We construct efficient nets on the sphere with lattice structure, and show that the lattice points typically have large RLCD. This allows us to derive strong anti-concentration properties for the distance between a fixed column of $A$ and the linear span of the remaining columns, and prove the main result.