Periodic orbits (equivalence classes of closed paths up to cyclic shifts) play an important role in applications of graph theory. For example, they appear in the definition of the Ihara zeta function and exact trace formulae for the spectra of quantum graphs. Circulant graphs are Cayley graphs of $\mathbb{Z}_n$. Here we consider directed Cayley graphs with two generators (2-regular Cayley digraphs). We determine the number of primitive periodic orbits of a given length (total number of directed edges) in terms of the number of times edges corresponding to each generator appear in the periodic orbit (the step count). Primitive periodic orbits are those periodic orbits that cannot be written as a repetition of a shorter orbit. We describe the lattice structure of lengths and step counts for which periodic orbits exist and characterize the repetition number of a periodic orbit by its winding number (the sum of the step sequence divided by the number of vertices) and the repetition number of its step sequence. To obtain these results, we also evaluate the number of Lyndon words on an alphabet of two letters with a given length and letter count.