Semantic communication focuses on conveying the task-relevant meaning rather than exact bitwise recovery. For image transmission with a generative receiver, relying only on text descriptions can be insufficient to preserve instance-specific visual evidence, whereas sending dense latent representations can incur substantial overhead. This paper presents a receiver-driven closed-loop scheme that transmits a short caption together with an initial sparse subset of latent blocks, and then uses feedback to request additional blocks only when needed. At each round, the receiver reconstructs the image via latent diffusion inpainting and applies a semantic consistency check between a caption generated from the reconstruction and the received caption, using a lightweight language similarity score such as ROUGE-L. The receiver stops early once a target consistency level is met, and otherwise requests a small number of additional latent blocks to refine the reconstruction. Experiments on Flickr30k over AWGN channels demonstrate a controllable rate-quality tradeoff. Adaptive feedback achieves the strongest semantic alignment and the lowest failure rate, outperforming budget-matched one-shot transmission while typically using fewer latent blocks than always-on retransmission.