We compare phase matching for four-wave mixing using one, two, three, and four waveguide modes. For the comparison, we use numerical optimizations and an estimate of the generated idler power. We present results for few-mode fibers and nano-rib waveguides and show that for both waveguide types, four-wave mixing bandwidths and idler powers are best for one- and two-mode operation and that four-mode four-wave mixing is not feasible at all. Some nano-rib waveguides support three-mode four-wave mixing, albeit with much reduced bandwidth and reduced idler power.