A secure optical communication requires both high transmission efficiency and high authentication performance, while existing cryptographic key distribution protocols based on ghost imaging have many shortcomings. Here, based on computational ghost imaging, we propose an interactive protocol that enables multi-party cryptographic key distribution over a public network and self-authentication by setting an intermediary that shares partial roles of the server. This fragment-synthesis-based authentication method may facilitate the remote distribution of cryptographic keys.