Combinatorial $t$-designs have nice applications in coding theory, finite geometries and several engineering areas. The objective of this paper is to study how to obtain $3$-designs with $2$-transitive permutation groups. The incidence structure formed by the orbits of a base block under the action of the general affine groups, which are $2$-transitive, is considered. A characterization of such incidence structure to be a $3$-design is presented, and a sufficient condition for the stabilizer of a base block to be trivial is given. With these general results, infinite families of $3$-designs are constructed by employing APN functions. Some $3$-designs presented in this paper give rise to self-dual binary codes or linear codes with optimal or best parameters known. Several conjectures on $3$-designs and binary codes are also presented.