Let $G$ be a graph on $[n]$ and $J_G$ be the binomial edge ideal of $G$ in the polynomial ring $S=\mathbb{K}[x_1,\ldots,x_n,y_1,\ldots,y_n]$. In this paper we investigate some topological properties of a poset associated to the minimal primary decomposition of $J_G$. We show that this poset admits some specific subposets which are contractible. This in turn, provides some interesting algebraic consequences. In particular, we characterize all graphs $G$ for which $\mathrm{depth}\hspace{1.2mm} S/J_G=4$.