Localization and tracking are two very active areas of research for robotics, automation, and the Internet-of-Things. Accurate tracking for a large number of devices usually requires deployment of substantial infrastructure (infrared tracking systems, cameras, wireless antennas, etc.), which is not ideal for inaccessible or protected environments. This paper stems from the challenge posed such environments: cover a large number of units spread over a large number of small rooms, with minimal required localization infrastructure. The idea is to accurately track the position of handheld devices or mobile robots, without interfering with its architecture. Using Ultra-Wide Band (UWB) devices, we leveraged our expertise in distributed and collaborative robotic systems to develop an novel solution requiring a minimal number of fixed anchors. We discuss a strategy to share the UWB network together with an Extended Kalman filter derivation to collaboratively locate and track UWB-equipped devices, and show results from our experimental campaign tracking visitors in the Chambord castle in France.