For groups without strict hierarchy, collective decisions often emerge through compromise. We develop a second-order network model of collective decision-making using a dissipative Hamiltonian formulation, in which informed agents introduce preferred directions while uninformed participants contribute only direction-free dissipation. We show that under low conflict, the model admits a locally unique, exponentially stable compromise state. Using a structured modular network we further show that as conflict increases the local compromise branch terminates through a saddle-node fold rather than through a smooth mean-field symmetry-breaking transition. Modular polarized states persist on branches that are locally separated from the compromise branch. Direction-free dissipation does not shift the static structural threshold, but it delays escape from the saddle-node ghost and pushes the observable onset of polarization to larger conflicts. Our work identifies a dissipation-mediated mechanism, complementary to connectivity-based accounts, through which uninformed participants stabilize collective behavior in biological and engineered swarms.