Driven by many applications in graph analytics, the problem of computing $k$-edge connected components ($k$-ECCs) of a graph $G$ for a user-given $k$ has been extensively studied recently. In this paper, we investigate the problem of constructing the hierarchy of edge connectivity-based graph decomposition, which compactly represents the $k$-ECCs of a graph for all possible $k$ values. This is based on the fact that each $k$-ECC is entirely contained in a $(k-1)$-ECC. In contrast to the existing approaches that conduct the computation either in a bottom-up or a top-down manner, we propose a binary search-based framework which invokes a $k$-ECC computation algorithm as a black box. Let $T_{kecc}(G)$ be the time complexity of computing all $k$-ECCs of $G$ for a specific $k$ value. We prove that the time complexity of our framework is ${\cal O}\big( (\log δ(G))\times T_{kecc}(G)\big)$, where $δ(G)$ is the degeneracy of $G$ and equals the maximum value among the minimum vertex degrees of all subgraphs of $G$. As $δ(G)$ is typically small for real-world graphs, this time complexity is optimal up to a logarithmic factor.