In the Visual Heritage domain, search engines are expected to support archaeologists and curators to address cross-correlation and searching across multiple collections. Archaeological excavations return artifacts that often are damaged with parts that are fragmented in more pieces or totally missing. The notion of similarity among fragments cannot simply base on the geometric shape but style, material, color, decorations, etc. are all important factors that concur to this concept. In this work, we discuss to which extent the existing techniques for 3D similarity matching are able to approach fragment similarity, what is missing and what is necessary to be further developed.