The Harborth constant of a finite group $G$ is the smallest integer $k\geq \exp(G)$ such that any subset of $G$ of size $k$ contains $\exp(G)$ distinct elements whose product is $1$. Generalizing previous work on the Harborth constants of dihedral groups, we compute the Harborth constants for the metacyclic groups of the form $H_{n, m}=\langle x, y \mid x^n=1, y^2=x^m, yx=x^{-1}y \rangle$. We also solve the "inverse" problem of characterizing all smaller subsets that do not contain $\exp(H_{n,m})$ distinct elements whose product is $1$.