The study of perfect state transfer on graphs has attracted a great deal of attention during the past ten years because of its applications to quantum information processing and quantum computation. Perfect state transfer is understood to be a rare phenomenon. This paper establishes necessary and sufficient conditions for a bi-Cayley graph having perfect state transfer over any given finite abelian group. As corollaries, many known and new results are obtained on Cayley graphs having perfect state transfer over abelian groups, (generalized) dihedral groups, semi-dihedral groups and generalized quaternion groups. Especially, we give an example of a connected non-normal Cayley graph over a dihedral group having perfect state transfer between two distinct vertices, which was thought impossible.