Network dismantling aims at breaking a network into disconnected components, and attacking vertices that intersect with many loops has proven to be a most efficient strategy. But the existing loop-focusing methods treat the short loops within densely connected local clusters (e.g., cliques) as equally important as the long loops connecting different clusters. Here we propose for highly clustered artificial and real-world networks a bipartite factor-graph formulation that retains all the long loops while simplifies the local dense clusters as individual factor nodes. We develop a mean-field theory for the associated long-loop feedback vertex set problem and apply its message-passing equation as a solver for network dismantling. The proposed factor-graph loop algorithm outperforms the current state-of-the-art graph loop algorithms by a considerable margin on various real networks. Further improvement in dismantling performance is achievable by optimizing the choice of the local dense clusters.