In 1992, Terwilliger introduced the notion of the \emph{Terwilliger algebra} in order to study association schemes. The Terwilliger algebra of an association scheme $\mathcal{A}$ is the subalgebra of the complex matrix algebra, generated by the \emph{Bose-Mesner algebra} of $\mathcal{A}$ and its dual idempotents with respect to a point $x$. In [{\em Kyushu Journal of Mathematics}, 49(1):93--102, 1995] Bannai and Munemasa determined the dimension of the Terwilliger algebra of abelian groups and dihedral groups, by showing that they are triply transitive (i.e., triply regular and dually triply regular). In this paper, we give a generalization of their results to the group association scheme of semidirect products of the form $C_n\rtimes C_2$, where $C_m$ is a cyclic group of order $m\geq 2$. Moreover, we will give the complete characterization of the Wedderburn components of the Terwilliger algebra of these groups.