We define cyclic $U$-statistics as a variant of $U$-statistics based on variables $X_1,\dots,X_n$ that are assumed to be cyclically ordered. We also define alternating $U$-statistics where in the definition terms are summed with alternating sings (in three different ways). Only $U$-statistics of order 2 are considered. The definitions are inspired by special cases studied by Chebikin (2008) and Even-Zohar (2017) for random permutations. We show limit theorems similar to well-known results for standard $U$-statistics, but with some differences between the different versions. In particular, we find both ``nondegenerate'' normal limits and ``degenerate'' non-normal limits.