Controlling complex physics systems is important in diverse domains. While diffusion-based methods have demonstrated advantages over classical model-based approaches and myopic sequential learning methods in achieving global trajectory consistency, they are limited by sample efficiency.This paper presents SEDC (Sample-Efficient Diffusion-based Control), a novel framework addressing core challenges in complex physics systems: high-dimensional state-control spaces, strong nonlinearities, and the gap between non-optimal training data and near-optimal control laws.Our approach introduces a novel control paradigm by architecturally decoupling state-control modeling and decomposing dynamics, while a guided self-finetuning process iteratively refines the control law towards optimality. We validate SEDC across diverse complex nonlinear systems, including high-dimensional fluid dynamics (Burgers), chaotic synchronization networks (Kuramoto), and real-world power grid stability control (Swing Equation). Our method achieves 39.5\%-47.3\% better control accuracy than state-of-the-art baselines while using only 10\% of the training samples. The implementation is available at \href{https://anonymous.4open.science/r/DIFOCON-C019}{here}.