We recall the notions of Clifford and Clifford-like parallelisms in a $3$-dimensional projective double space. In a previous paper the authors proved that the linear part of the full automorphism group of a Clifford parallelism is the same for all Clifford-like parallelisms which can be associated to it. In this paper, instead, we study the action of such group on parallel classes thus achieving our main results on characterisation of the Clifford parallelisms among Clifford-like ones.