Let $A$ be an $N\times n$ random matrix whose entries are coordinates of an isotropic log-concave random vector in $\mathbb{R}^{Nn}$. We prove sharp lower tail estimates for the smallest singular value of $A$ in the following cases: (1) when $N=n$ and $A$ is drawn from an unconditional distribution, with no independence assumption; (2) when the columns of $A$ are independent and $N\geq n$; (3) when $A$ is sufficiently tall, that is $N\geq (1+λ)n$ for any positive constant $λ$.