The set of hook lengths of an integer partition $λ$ is the complement of some numerical semigroup $S$. There has been recent interest in studying the number of partitions with a given set of hook lengths. Very little is known about the distribution of sizes of this finite set of partitions. We focus on the problem of determining the size of the smallest partition with its set of hook lengths equal to $\mathbb{N}\setminus S$.