We provide a theoretical, numerical, and experimental investigation of the Kerr nonlinearity impact on the performance of a frequency-multiplexed Extreme Learning Machine (ELM). In such ELM, the neuron signals are encoded in the lines of a frequency comb. The Kerr nonlinearity facilitates the randomized neuron connections allowing for efficient information mixing. A programmable spectral filter applies the output weights. The system operates in a continuous-wave regime. Even at low input peak powers, the resulting weak Kerr nonlinearity is sufficient to significantly boost the performance on several tasks. This boost already arises when one uses only the very small Kerr nonlinearity present in a 20-meter long erbium-doped fiber amplifier. In contrast, a subsequent propagation in 540 meters of a single-mode fiber improves the performance only slightly, whereas additional information mixing with a phase modulator does not result in a further improvement at all. We introduce a model to show that, in frequency-multiplexed ELMs, the Kerr nonlinearity mixes information via four-wave mixing, rather than via self- or cross-phase modulation. At low powers, this effect is quartic in the comb-line amplitudes. Numerical simulations validate our experimental results and interpretation.