The space of $4 \times 4$ special orthogonal matrices with zeros on the diagonal decomposes into the union of $14$ irreducible surfaces whose intersections are beautifully encoded by the cuboctahedron. Using this decomposition, we exhibit a totally real witness set for $SO(4)$. We explain how to obtain a similar decomposition for $SO(5)$, where the $64$ components can be grouped to obtain such a correspondence with the face lattice of a $3$-polytope. We show that no such pattern exists for $SO(6)$.