Undersampling can accelerate the signal acquisition but at the cost of bringing in artifacts. Removing these artifacts is a fundamental problem in signal processing and this task is also called signal reconstruction. Through modeling signals as the superimposed exponential functions, deep learning has achieved fast and high-fidelity signal reconstruction by training a mapping from the undersampled exponentials to the fully sampled ones. However, the mismatch, such as the sampling rate of undersampling, the organ and the contrast of imaging, between the training and target data will heavily compromise the reconstruction. To address this issue, we propose Alternating Deep Low-Rank (ADLR), which combines deep learning solvers and classic optimization solvers. Experiments on the reconstruction of synthetic and realistic biomedical magnetic resonance signals demonstrate that ADLR can effectively mitigate the mismatch issue and achieve lower reconstruction errors than state-of-the-art methods.