Compact finite-rank nilspaces have become central in the nilspace approach to higher-order Fourier analysis, notably through their role in a general form of the inverse theorem for the Gowers norms. This paper studies these nilspaces per se, and in connection with further refinements of this inverse theorem that have been conjectured recently. Our first main result states that every compact finite-rank nilspace is obtained by taking a free nilspace (a nilspace based on an abelian group of the form $\mathbb{Z}^{r}\times \mathbb{R}^s$) and quotienting this by a discrete group action of a specific type, describable in terms of polynomials. We call these group actions "higher-order lattice actions", as they generalize actions of lattices in $\mathbb{Z}^r\times \mathbb{R}^s$. The second main result (which relies on the first one) represents every compact finite-rank nilspace as a double-coset space $K\backslash G / Γ$ where $G$ is a nilpotent Lie group of a specific kind. Our third main result extends the aforementioned results to $k$-step compact nilspaces (not necessarily of finite rank), by representing any such nilspace as a quotient of infinite products of free nilspaces and also as double coset spaces $K\backslash G/Γ$ where $G$ is a degree-$k$ nilpotent pro-Lie group. These results open the study of compact nilspaces to areas more classical than nilspace theory, such as the theory of topological group actions. The results also require developing the theory of topological non-compact nilspaces, for which we provide groundwork in this paper. Applications include new inverse theorems for Gowers norms on any finite abelian group. These theorems are purely group theoretic in that the correlating harmonics are based on double-coset spaces. This yields progress towards the Jamneshan-Tao conjecture.