The convergence of Terahertz (THz) communications and Federated Learning (FL) promises ultra-fast distributed learning, yet the impact of realistic wideband impairments on optimization dynamics remains theoretically uncharacterized. This paper bridges this gap by developing a multicarrier stochastic framework that explicitly couples local gradient updates with frequency-selective THz effects, including beam squint, molecular absorption, and jitter. Our analysis uncovers a critical diversity trap: under standard unbiased aggregation, the convergence error floor is driven by the harmonic mean of subcarrier SNRs. Consequently, a single spectral hole caused by severe beam squint can render the entire bandwidth useless for reliable model updates. We further identify a fundamental bandwidth limit, revealing that expanding the spectrum beyond a critical point degrades convergence due to the integration of thermal noise and gain collapse at band edges. Finally, we demonstrate that an SNR-weighted aggregation strategy is necessary to suppress the variance singularity at these spectral holes, effectively recovering convergence in high-squint regimes where standard averaging fails. Numerical results validate the expected impact of the discussed physical layer parameters' on performance of THz-FL systems.