The study of feedback control inspired by Maxwell's demon is central to the understanding of the relationship between thermodynamics and information. In this paper, we establish fundamental lower limits on the work costs of system conversion with quantum feedback, where quantum side information acquired in advance can be fed back to the system coherently by a controller. From two basic operational principles that every physically admissible feedback-control scheme should satisfy, we derive the tightest possible bounds on the single-shot work of formation and extractable work of an arbitrary quantum system given arbitrary quantum side information held by the controller. These bounds are expressed in terms of information measures simultaneously generalizing conditional entropies, mutual informations, and relative entropies. In the asymptotic limit, they lead to a generalized second law of thermodynamics with quantum feedback, featuring a conditional Helmholtz free energy. Our findings also provide precise thermodynamic meanings for the negativity of single-shot conditional entropies and resolve an open problem in the axiomatic reconstruction of such conditional entropies.