Energy harvesting aided mobile edge computing (MEC) has gained much attention for its widespread application in the computation-intensive, latency-sensitive and energy-hungry scenario. In this paper, computation offloading from multi-MD to multi-MEC-s in heterogeneous MEC systems with energy harvesting is investigated from a game theoretic perspective. The objective is to minimize the average response time of an MD that consists of communication time, waiting time and processing time. M/G/1 queueing models are established for MDs and MEC-ss. The interference among MDs, the randomness in computation task generation, harvested energy arrival, wireless channel state, queueing at the MEC-s, and the power budget constraint of an MD are taken into consideration. A noncooperative computation offloading game is formulated. We give the definition and existence analysis of the Nash equilibrium (NE). Furthermore, we reconstruct the optimization problem of an MD. A 2-step decomposition is presented and performed. Thereby, we arrive at a one-dimensional search problem and a greatly shrunken sub-problem. We can obtain the optimal solution of the sub-problem by seeking the finite solutions of its Karush-Kuhn-Tucker (KKT) conditions. Thereafter, a distributive NE-orienting iterated best-response algorithm is designed. Simulations are carried out to illustrate the convergence performance and parameter effect.