In 1990 Erdős, Faudree, Rousseau and Schelp proved that for $k\geq 2$, every graph with $n\geq k+1$ vertices and $(k-1)(n-k+2)+\binom{k-2}{2}+1$ edges contains a subgraph of minimum degree $k$ on at most $n-\sqrt{n}/\sqrt{6k^3}$ vertices. They conjectured that it is possible to remove at least $ε_k n$ many vertices and remain with a subgraph of minimum degree $k$, for some $ε_k>0$. We make progress towards their conjecture by showing that one can remove at least $Ω(n/\log n)$ many vertices.