This paper investigates a cellular edge caching design under an extremely large number of small base stations (SBSs) and users. In this ultra-dense edge caching network (UDCN), SBS-user distances shrink, and each user can request a cached content from multiple SBSs. Unfortunately, the complexity of existing caching controls' mechanisms increases with the number of SBSs, making them inapplicable for solving the fundamental caching problem: How to maximize local caching gain while minimizing the replicated content caching? Furthermore, spatial dynamics of interference is no longer negligible in UDCNs due to the surge in interference. In addition, the caching control should consider temporal dynamics of user demands. To overcome such difficulties, we propose a novel caching algorithm weaving together notions of mean-field game theory and stochastic geometry. These enable our caching algorithm to become independent of the number of SBSs and users, while incorporating spatial interference dynamics as well as temporal dynamics of content popularity and storage constraints. Numerical evaluation validates the fact that the proposed algorithm reduces not only the long run average cost by at least 24% but also the number of replicated content by 56% compared to a popularity-based algorithm.