Social infrastructure plays a critical role in shaping neighborhood well-being by fostering social and cultural interaction, enabling service provision, and encouraging exposure to diverse environments. Despite the growing knowledge of its spatial accessibility, time use at social infrastructure places is underexplored due to the lack of a spatially resolved national dataset. We address this gap by developing scalable Social-Infrastructure Time Use measures (STU) that capture length and depth of engagement, activity diversity, and spatial inequality, supported by first-of-their-kind datasets spanning multiple geographic scales from census tracts to metropolitan areas. Our datasets leverage anonymized and aggregated foot traffic data collected between 2019 and 2024 across 49 continental U.S. states. The data description reveals variances in STU across time, space, and differing neighborhood sociodemographic characteristics. Validation demonstrates generally robust population representation, consistent with established national survey findings while revealing more nuanced patterns. Future analyses could link STU with public health outcomes and environmental factors to inform targeted interventions aimed at enhancing population well-being and guiding social infrastructure planning and usage.