In network management, swiftly and accurately identifying traffic anomalies, including Distributed Denial-of-Service (DDoS) attacks and unexpected network disruptions, is essential for network stability and security. Key to this process is the detection of Hierarchical Heavy Hitters (HHH), which significantly aids in the management of high-speed IP traffic. This study introduces ResidualSketch, a novel algorithm for HHH detection in hierarchical traffic analysis. ResidualSketch distinguishes itself by incorporating Residual Blocks and Residual Connections at crucial layers within the IP hierarchy, thus mitigating the Gradual Error Diffusion (GED) phenomenon in previous methods and reducing memory overhead while maintaining low update latency. Through comprehensive experiments on various datasets, we demonstrate that ResidualSketch outperforms existing state-of-the-art solutions in terms of accuracy and update speed across multiple layers of the network hierarchy. All related codes of ResidualSketch are open-source at GitHub.