This paper develops a general framework for multi-agent control synthesis, which applies to a wide range of problems with convergence guarantees, including those with time-varying objective functions. The proposed framework achieves decentralization without inducing dynamical coupling among agents, and it naturally supports multi-objective robotics and real-time implementation. To demonstrate its generality and effectiveness, the framework is applied to solve three representative problems, namely time-varying leader-follower formation control, decentralized coverage control for time-varying density functions without approximations, which is a long-standing open problem, and safe formation navigation in a dense environment.