Cyclic codes are a subclass of linear codes and have wide applications in data storage systems, communication systems and consumer electronics due to their efficient encoding and decoding algorithms. Let $p\ge 5$ be an odd prime and $m$ be a positive integer. Let $\mathcal{C}_{(1,e,s)}$ denote the $p$-ary cyclic code with three nonzeros $α$, $α^e$, and $α^s$, where $α$ is a generator of ${\mathbb F}_{p^m}^*$, $s=\frac{p^m-1}{2}$, and $2\le e\le p^m-2$. In this paper, we present four classes of optimal $p$-ary cyclic codes $\mathcal{C}_{(1,e,s)}$ with parameters $[p^m-1,p^m-2m-2,4]$ by analyzing the solutions of certain polynomials over finite fields. Some previous results about optimal quinary cyclic codes with parameters $[5^m-1,5^m-2m-2,4]$ are special cases of our constructions. In addition, by analyzing the irreducible factors of certain polynomials over ${\mathbb F}_{5^m}$, we present two classes of optimal quinary cyclic codes $\mathcal{C}_{(1,e,s)}$.