This paper focuses on edge caching in dense heterogeneous cellular networks (HetNets), in which small base stations (SBSs) with limited cache size store the popular contents, and massive multiple-input multiple-output (MIMO) aided macro base stations provide wireless self-backhaul when SBSs require the non-cached contents. Our aim is to address the effects of cell load and hit probability on the successful content delivery (SCD), and present the minimum required base station density for avoiding the access overload in an arbitrary small cell and backhaul overload in an arbitrary macrocell. The massive MIMO backhaul achievable rate without downlink channel estimation is derived to calculate the backhaul time, and the latency is also evaluated in such networks. The analytical results confirm that hit probability needs to be appropriately selected, in order to achieve SCD. The interplay between cache size and SCD is explicitly quantified. It is theoretically demonstrated that when non-cached contents are requested, the average delay of the non-cached content delivery could be comparable to the cached content delivery with the help of massive MIMO aided self-backhaul, if the average access rate of cached content delivery is lower than that of self-backhauled content delivery. Simulation results are presented to validate our analysis.