In this paper, we consider the problem on the existence of perfect state transfer(PST for short) on semi-Cayley graphs over abelian groups (which are not necessarily regular), i.e on the graphs having semiregular and abelian subgroups of automorphisms with two orbits of equal size. We stablish a characterization of semi-Cayley graphs over abelian groups having PST. As a result, we give a characterization of Cayley graphs over groups with an abelian subgroup of index 2 having PST, which improves the earlier results on Cayley graphs over abelian groups, dihedral groups and dicyclic group and determines Cayley graphs over generalized dihedral groups and generalized dicyclic groups having PST.