In (fully) dynamic set cover, the goal is to maintain an approximately optimal solution to a dynamically evolving instance of set cover, where in each step either an element is added to or removed from the instance. The two main desiderata of a dynamic set cover algorithm are to minimize at each time-step, the recourse, which is the number of sets removed from or added to the solution, and the update time to compute the updated solution. This problem has been extensively studied over the last decade leading to many results that achieve ever-improving bounds on the recourse and update time, while maintaining a solution whose cost is comparable to that of offline approximation algorithms. In this paper, we give the first algorithms to simultaneously achieve non-trivial worst-case bounds for recourse and update time. Specifically, we give fully-dynamic set cover algorithms that simultaneously achieve $O(\log n)$ recourse and $f\cdot \textrm{poly}\log(n)$ update time in the worst-case, for both approximation regimes: $O(\log n)$ and $O(f)$ approximation. (Here, $n, f$ respectively denote the maximum number of elements and maximum frequency of an element across all instances.) Prior to our work, all results for this problem either settled for amortized bounds on recourse and update time, or obtained $f\cdot \textrm{poly}\log(n)$ update time in the worst-case but at the cost of $Ω(m)$ worst-case recourse. (Here, $m$ denotes the number of sets. Note that any algorithm has recourse at most $m$.)