In this paper, the edge caching problem in ultra-dense fog radio access networks (F-RAN) is investigated. Taking into account time-variant user requests and ultra-dense deployment of fog access points (F-APs), we propose a dynamic distributed edge caching scheme to jointly minimize the request service delay and fronthaul traffic load. Considering the interactive relationship among F-APs, we model the caching optimization problem as a stochastic differential game (SDG) which captures the temporal dynamics of F-AP states and incorporates user requests status. The SDG is further approximated as a mean field game (MFG) by exploiting the ultra-dense property of F-RAN. In the MFG, each F-AP can optimize its caching policy independently through iteratively solving the corresponding partial differential equations without any information exchange with other F-APs. The simulation results show that the proposed edge caching scheme outperforms the baseline schemes under both static and time-variant user requests.