The use of game theoretic models has been quite successful in describing various cooperative and non-cooperative optimization problems in networks and other domains of computer systems. In this paper, we study an application of game theoretic models in the domain of distributed system, where nodes play a game to balance the total processing loads among themselves. We have used congestion gaming model, a model of game theory where many agents compete for allocating resources, and studied the existence of Nash Equilibrium for such types of games. As the classical congestion game is known to be PLS-Complete, we use an approximation, called the ε-Congestion game, which converges to ε-Nash equilibrium within finite number of steps under selected conditions. Our focus is to define the load balancing problem using the model of ε-congestion games, and finally provide a greedy algorithm for load balancing in distributed systems. We have simulated our proposed system to show the effect of ε-congestion game, and the distribution of load at equilibrium state.