Dynamic Metasurface Antennas (DMAs) constitute a promising solution for extremely large antenna arrays, requiring lower power consumption and reduced hardware cost as compared to conventional phased arrays. In this paper, we consider a cell-free Orthogonal Frequency Division Multiplexing (OFDM) system comprising multiple Base Stations (BSs) equipped with parallel-plate-waveguided DMAs, which aims to serve multiple users in the downlink direction. Focusing on a realistic frequency-selective model for the response-tunable elements of each DMA panel, and targeting to surpass the necessity of centralized designs that rely on a central processing unit with high computational power, we present a distributed optimization framework with minimal control information exchange for the frequency-selective analog and digital beamforming matrices of the multiple BSs, having the system spectral efficiency maximization as the design objective. Considering imperfect Channel State Information (CSI) availability at each BS, we devise a parallel decomposition framework for the configuration of the tunable parameters of each DMA-based BS. Our numerical results showcase the robustness of the proposed distributed beamforming design over different CSI conditions, and quantify the critical role of taking into account mutual coupling during the DMA design process.