For a digraph $Γ$, if $F$ is the smallest field that contains all roots of the characteristic polynomial of the adjacency matrix of $Γ$, then $F$ is called the splitting field of $Γ$. The extension degree of $F$ over the field of rational numbers $\mathbb{Q}$ is said to be the algebraic degree of $Γ$. A digraph is a semi-Cayley digraph over a group $G$ if it admits $G$ as a semiregular automorphism group with two orbits of equal size. A semi-Cayley digraph $\mathrm{SC}(G,T_{11},T_{22},T_{12},T_{21})$ is called quasi-abelian if each of $T_{11},T_{22},T_{12}$ and $T_{21}$ is a union of some conjugacy classes of $G$. This paper determines the splitting field and the algebraic degree of a quasi-abelian semi-Cayley digraph over any finite group in terms of irreducible characters of groups. This work generalizes the previous works on algebraic degrees of Cayley graphs over abelian groups and any group having a subgroup of index 2, and semi-Cayley digraphs over abelian groups.