The Terwilliger algebras of association schemes over an arbitrary field $\mathbb{F}$ were called the Terwilliger $\mathbb{F}$-algebras of association schemes in [8]. In this paper, we study the Terwilliger $\mathbb{F}$-algebras of factorial association schemes. We determine the $\mathbb{F}$-dimensions, the centers, the semisimplicity, the Jacobson radicals, and the algebraic structures of the Terwilliger $\mathbb{F}$-algebras of factorial association schemes.