Regarding neighbor balance, we consider natural generalizations of $D$-complete Latin squares and Vatican squares from the finite to the infinite. We show that if $G$ is an infinite abelian group with $|G|$-many square elements, then it is possible to permute the rows and columns of the Cayley table to create an infinite Vatican square. We also construct a Vatican square of any given infinite order that is not obtainable by permuting the rows and columns of a Cayley table. Regarding orthogonality, we show that every infinite group $G$ has a set of $|G|$ mutually orthogonal orthomorphisms and hence there is a set of $|G|$ mutually orthogonal Latin squares based on $G$. We show that an infinite group $G$ with $|G|$-many square elements has a strong complete mapping; and, with some possible exceptions, infinite abelian groups have a strong complete mapping.