We consider the challenging problem of tracking multiple objects using a distributed network of sensors. In the practical setting of nodes with limited field of views (FoVs), computing power and communication resources, we develop a novel distributed multi-object tracking algorithm. To accomplish this, we first formalise the concept of label consistency, determine a sufficient condition to achieve it and develop a novel \textit{label consensus approach} that reduces label inconsistency caused by objects' movements from one node's limited FoV to another. Second, we develop a distributed multi-object fusion algorithm that fuses local multi-object state estimates instead of local multi-object densities. This algorithm: i) requires significantly less processing time than multi-object density fusion methods; ii) achieves better tracking accuracy by considering Optimal Sub-Pattern Assignment (OSPA) tracking errors over several scans rather than a single scan; iii) is agnostic to local multi-object tracking techniques, and only requires each node to provide a set of estimated tracks. Thus, it is not necessary to assume that the nodes maintain multi-object densities, and hence the fusion outcomes do not modify local multi-object densities. Numerical experiments demonstrate our proposed solution's real-time computational efficiency and accuracy compared to state-of-the-art solutions in challenging scenarios. We also release source code at https://github.com/AdelaideAuto-IDLab/Distributed-limitedFoV-MOT for our fusion method to foster developments in DMOT algorithms.