A graph is said to be uniquely hamiltonian if it has a unique hamiltonian cycle. For a natural extension of this concept to infinite graphs, we find all uniquely hamiltonian vertex-transitive graphs with finitely many ends, and also discuss some examples with infinitely many ends. In particular, we show each nonabelian free group $F_n$ has a Cayley graph of degree $2n + 2$ that has a unique hamiltonian circle. (A weaker statement had been conjectured by A. Georgakopoulos.) Furthermore, we prove that these Cayley graphs of $F_n$ are outerplanar.