A fundamental question of human society is the evolution of cooperation. Many previous studies explored this question via setting spatial background, where players obtain their payoffs by playing game with their nearest neighbors. Another undoubted fact is that environment plays an important role in the individual development. Inspired by these phenomena, we reconsider the definition of individual fitness which integrates the environment, denoted by the average payoff of all individual neighbors, with the traditional individual payoffs by introducing a selection parameter $u$. Tuning $u$ equal to zero returns the traditional version, while increasing $u$ bears the influence of environment. We find that considering the environment, i.e. integrating neighborhoods in the evaluation of fitness, promotes cooperation. If we enhance the value of $u$, the invasion of defection could be resisted better. We also provide quantitative explanations and complete phase diagrams presenting the influence of environment on the evolution of cooperation. Finally, the universality of this mechanism is testified for different neighborhood sizes, different topology structures and different game models. Our work may shed a light on the emergence and persistence of cooperation in our life.