We show that there is a set which is not a set of multiple recurrence despite being a set of recurrence for nil-Bohr sets. This answers Huang, Shao, and Ye's \enquote{higher-order} version of Katznelson's Question on Bohr recurrence and topological recurrence in the negative. Equivalently, we construct a set $S$ so that there is a finite coloring of $\mathbb{N}$ without three-term arithmetic progressions with common differences in $S$, but so that $S$ lacks the usual polynomial obstacles to arithmetic progressions.