The difference graph $D(G)$ of a finite group $G$ is the difference of enhanced power graph of $G$ and power graph of $G$, with all isolated vertices are removed. In this paper we study the connectedness and perfectness of $D(G)$ with respect to various properties of the underlying group $G$. We also find several connection between the difference graph of $G$ and the Gruenberg-Kegel graph of $G$.