The clutter in the ground-penetrating radar (GPR) radargram disguises or distorts subsurface target responses, which severely affects the accuracy of target detection and identification. Existing clutter removal methods either leave residual clutter or deform target responses when facing complex and irregular clutter in the real-world radargram. To tackle the challenge of clutter removal in real scenarios, a clutter-removal neural network (CR-Net) trained on a large-scale hybrid dataset is presented in this study. The CR-Net integrates residual dense blocks into the U-Net architecture to enhance its capability in clutter suppression and target reflection restoration. The combination of the mean absolute error (MAE) loss and the multi-scale structural similarity (MS-SSIM) loss is used to effectively drive the optimization of the network. To train the proposed CR-Net to remove complex and diverse clutter in real-world radargrams, the first large-scale hybrid dataset named CLT-GPR dataset containing clutter collected by different GPR systems in multiple scenarios is built. The CLT-GPR dataset significantly improves the generalizability of the network to remove clutter in real-world GPR radargrams. Extensive experimental results demonstrate that the CR-Net achieves superior performance over existing methods in removing clutter and restoring target responses in diverse real-world scenarios. Moreover, the CR-Net with its end-to-end design does not require manual parameter tuning, making it highly suitable for automatically producing clutter-free radargrams in GPR applications. The CLT-GPR dataset and the code implemented in the paper can be found at https://haihan-sun.github.io/GPR.html.