Accurate electromagnetic field (EMF) exposure mapping is critical for wireless network planning, environmental monitoring, and the deployment of next generation communication systems. The mapping results can be converted into the form of a radio map, a key technology in digital twin communication systems, used to describe the wireless signal propagation characteristics at every location in a specific area. Existing deep learning approaches treat propagation estimation as a pure regression problem and do not enforce physical consistency in the predicted fields. In this paper, we propose Phy2-ExposNet, a novel neural network framework that decouples exposure mapping into a physics-informed estimation stage and a transformer-based residual refinement stage. It first estimates the fields under two physical constraints and then refines the resulting exposure map by capturing long range interactions and complex spatial propagation patterns. Experiments demonstrate that the proposed method achieves lower estimation error while significantly reducing model complexity compared to existing approaches. It achieves around 15% relative error reduction over strong baselines, while using over 80% fewer parameters than conventional physics-informed models. Ablation results further reveal that the physics-informed design is crucial for capturing complex propagation effects, particularly in boundary and shadow regions.