As a result of the recent popularity of social networks and the increase in the number of research papers published across all fields, attributed networks consisting of relationships between objects, such as humans and the papers, that have attributes are becoming increasingly large. Therefore, various studies for clustering attributed networks into sub-networks are being actively conducted. When clustering attributed networks using spectral clustering, the clustering accuracy is strongly affected by the quality of the similarity matrices, which are input into spectral clustering and represent the similarities between pairs of objects. In this paper, we aim to increase the accuracy by refining the matrices before applying spectral clustering to them. We verify the practicability of our proposed method by comparing the accuracy of spectral clustering with similarity matrices before and after refining them.