Radio Environment Maps (REMs) have the potential to serve as an important enabler for intelligent modeling and control in emerging AI-native 6G networks. Despite significant progress, most REM construction methods remain passive, relying on interpolation or static uncertainty models and lacking an explicit mechanism to reason about how future measurements will affect reconstruction quality under a limited measurement budget. In this paper, we formulate REM construction as a sequential decision-making problem and propose a world-model-inspired framework for active Received Signal Strength Indicator (RSSI) map reconstruction. By learning an internal representation of the radio environment and employing a dreaming mechanism to simulate the impact of candidate measurements, the proposed approach actively selects measurement locations under a limited budget. Experimental results on real indoor RSSI data demonstrate that the proposed method significantly outperforms Gaussian Process-based interpolation in the few-shot regime, achieving up to a fivefold reduction in Root Mean Square Error (RMSE) with the same number of measurements. These results highlight the potential of world models as a powerful paradigm for sample-efficient radio environment mapping and intelligent model-based sensing in 6G and beyond networks.