The Lovász complex $L(G)$ of a graph $G$ is a deformation retract of its neighborhood complex, equipped with a canonical $Z_2$-action. We show that, under mild assumptions, $L(G)$ is homeomorphic to a surface if and only if $G$ is a non-bipartite quadrangulation of the orbit space $L(G)/Z_2$ in which every $4$-cycle is facial. This yields a classification of the Lovász complexes of all such quadrangulations. As an application, we contextualize a result of Archdeacon \emph{et al.}\ and Mohar and Seymour on the chromatic number of quadrangulations, obtaining a stronger statement about the $Z_2$-index.