In this paper, we propose a fully distributed algorithm for frequency offsets estimation in decentralized systems. With the proposed algorithm, each node estimates its frequency offsets by local computations and limited exchange of information with its direct neighbors. Such algorithm does not require any centralized information processing or knowledge of global network topology. It is shown analytically that the proposed algorithm always converges to the optimal estimates regardless of network topology. Simulation results demonstrate the fast convergence of the algorithm and show that estimation mean-squared-error at each node touches the centralized Cramér-Rao bound within a few {iterations of message exchange}. Therefore, the proposed method has low overhead and is scalable with network size.