In literatures, there are various constructions of frequency hopping sequence (FHS for short) sets with good Hamming correlations. Some papers employed only multiplicative groups of finite fields to construct FHS sets, while other papers implicitly used only additive groups of finite fields for construction of FHS sets. In this paper, we make use of both multiplicative and additive groups of finite fields simultaneously to present a construction of optimal FHS sets. The construction provides a new family of optimal $\left(q^m-1,\frac{q^{m-t}-1}{r},rq^t;\frac{q^{m-t}-1}{r}+1\right)$ frequency hopping sequence sets archiving the Peng-Fan bound. Thus, the FHS sets constructed in literatures using either multiplicative groups or additive groups of finite fields are all included in our family. In addition, some other FHS sets can be obtained via the well-known recursive constructions through one-coincidence sequence set.