In this paper, it is shown that the syndromes of generalized Reed-Solomon (GRS) codes and alternant codes can be characterized in terms of inverse fast Fourier transform, regardless of code definitions. Then a fast decoding algorithm is proposed, which has a computational complexity of $O(n\log(n-k) + (n-k)\log^2(n-k))$ for all $(n,k)$ GRS codes and $(n,k)$ alternant codes. Particularly, this provides a new decoding method for Goppa codes, which is an important subclass of alternant codes. When decoding the binary Goppa code with length $8192$ and correction capability $128$, the new algorithm is nearly 10 times faster than traditional methods. The decoding algorithm is suitable for the McEliece cryptosystem, which is a candidate for post-quantum cryptography techniques.