Deep learning (DL) techniques have demonstrated strong performance in compressing and reconstructing channel state information (CSI) while reducing feedback overhead in massive MIMO systems. A key challenge, however, is their reliance on extensive site-specific training data, whose real-world collection incurs significant overhead and limits scalability across deployment sites. To address this, we propose leveraging site-specific digital twins to assist the training of DL-based CSI compression models. The digital twin integrates an electromagnetic (EM) 3D model of the environment, a hardware model, and ray tracing to produce site-specific synthetic CSI data, allowing DL models to be trained without the need for extensive real-world measurements. We further develop a fidelity analysis framework that decomposes digital twin quality into four key aspects: 3D geometry, material properties, ray tracing, and hardware modeling. We explore how these factors influence the reliability of the data and model performance. To enhance the adaptability to real-world environments, we propose a refinement strategy that incorporates a limited amount of real-world data to fine-tune the DL model pre-trained on the digital twin dataset. Evaluation results show that models trained on site-specific digital twins outperform those trained on generic datasets, with the proposed refinement method effectively enhancing performance using limited real-world data. The simulations also highlight the importance of digital twin fidelity, especially in 3D geometry, ray tracing, and hardware modeling, for improving CSI reconstruction quality. This analysis framework offers valuable insights into the critical fidelity aspects, and facilitates more efficient digital twin development and deployment strategies for various wireless communication tasks.