This paper proposes a new distributed algorithm for solving linear systems associated with a sparse graph under a generalised diagonal dominance assumption. The algorithm runs iteratively on each node of the graph, with low complexities on local information exchange between neighbouring nodes, local computation and local storage. For an acyclic graph under the condition of diagonal dominance, the algorithm is shown to converge to the correct solution in a finite number of iterations, equalling the diameter of the graph. For a loopy graph, the algorithm is shown to converge to the correct solution asymptotically. Simulations verify that the proposed algorithm significantly outperforms the classical Jacobi method and a recent distributed linear system solver based on average consensus and orthogonal projection.