Micro-architectural attacks use information leaked through shared resources to break hardware-enforced isolation. These attacks have been used to steal private information ranging from cryptographic keys to privileged Operating System (OS) data in devices ranging from mobile phones to cloud servers. Most existing software countermeasures either have unacceptable overheads or considerable false positives. Further, they are designed for specific attacks and cannot readily adapt to new variants. In this paper, we propose a framework called LEASH, which works from the OS scheduler to stymie micro-architectural attacks with minimal overheads, negligible impact of false positives, and is capable of handling a wide range of attacks. LEASH works by starving maliciously behaving threads at runtime, providing insufficient time and resources to carry out an attack. The CPU allocation for a falsely flagged thread found to be benign is boosted to minimize overheads. To demonstrate the framework, we modify Linux's Completely Fair Scheduler with LEASH and evaluate it with seven micro-architectural attacks ranging from Meltdown and Rowhammer to a TLB covert channel. The runtime overheads are evaluated with a range of real-world applications and found to be less than 1% on average.