Lights Out is a game which can be played on any graph $G$. Initially we have a configuration which assigns one of the two states on or off to each vertex. The aim of the game is to turn all vertices to off state for an initial configuration by activating some vertices where each activation switches the state of the vertex and all of its neighbors. If the aim of the game can be accomplished for all initial configurations then $G$ is called always solvable. We call the dimension of the kernel of the closed neighborhood matrix of the graph over the field $\mathbb{Z}_2$, nullity of $G$. It turns out that $G$ is always solvable if and only if its nullity is zero. Moreover, the number of solutions of a given configuration is also determined by the nullity. We investigate the problem of how nullity changes when an edge is added to or removed from a graph. As a result we show that for every graph with positive nullity there exists an edge whose removal decreases the nullity. Conversely, we show that for every always solvable graph which is not an even graph with odd order, there exists an edge whose addition increases the nullity. We also show that if an always solvable graph is not even, then there is an edge whose removal increases the nullity.