Self-reciprocal and self-conjugate-reciprocal polynomials over finite fields have been of interest due to their rich algebraic structures and wide applications. Self-reciprocal irreducible monic factors of $x^n-1$ over finite fields and their applications have been quite well studied. In this paper, self-conjugate-reciprocal irreducible monic (SCRIM) factors of $x^n-1$ over finite fields of square order have been focused on. The characterization of such factors is given together the enumeration formula. In many cases, recursive formulas for the number of SCRIM factors of $x^n-1$ have been given as well. As applications, Hermitian complementary dual codes over finite fields and Hermitian self-dual cyclic codes over finite chain rings of prime characteristic have been discussed.