Two doubly indexed families of polynomials in several indeterminates are considered. They are related to the falling and rising factorials in a similar way as the potential polynomials (introduced by L. Comtet) are related to the ordinary power function. We study the inversion relations valid for these factorial polynomials as well as the number families associated with them.