In opinion dynamics, how to model the enduring fragmentation phenomenon (disagreement, cleavage, and polarization) of social opinions has long possessed a central position. It is widely known that the confidence-based opinion dynamics provide an acceptant mechanism to produce fragmentation phenomenon. In this study, taking the famous confidence-based Hegselmann-Krause (HK) model, we examine the robustness of the fragmentation coming from HK dynamics and its variations with prejudiced and stubborn agents against random noise. Prior to possible insightful explanations, the theoretical results in this paper explicitly reveal that the well-appearing fragmentation of HK dynamics and its homogeneous variations finally vanishes in the presence of arbitrarily tiny noise, while only the HK model with heterogenous prejudices displays a solid cleavage in noisy environment.