Leveraging received signal strength (RSS) measurements for indoor localization is highly attractive due to their inherent availability in ubiquitous wireless protocols. However, prevailing RSS-based methods often depend on complex computational algorithms or specialized hardware, rendering them impractical for low-cost access points. To address these challenges, this paper introduces buffer-aided RMSProp (BARProp), a fast and memory-efficient localization algorithm specifically designed for RSS-based tasks. The key innovation of BARProp lies in a novel mechanism that dynamically adapts the decay factor by monitoring the energy variations of recent gradients stored in a buffer, thereby achieving both accelerated convergence and enhanced stability. Furthermore, BARProp requires less than 15% of the memory used by state-of-the-art methods. Extensive evaluations with real-world data demonstrate that BARProp not only achieves higher localization accuracy but also delivers at least a fourfold improvement in convergence speed compared to existing benchmarks.