Incorporating auxiliary information alongside primary data can significantly enhance the accuracy of simultaneous inference. However, existing multiple testing methods face challenges in efficiently incorporating complex side information, especially when it differs in dimension or structure from the primary data, such as network side information. This paper introduces a locally adaptive structure learning algorithm (LASLA), a flexible framework designed to integrate a broad range of auxiliary information into the inference process. Although LASLA is specifically motivated by the challenges posed by network-structured data, it also proves highly effective with other types of side information, such as spatial locations and multiple auxiliary sequences. LASLA employs a $p$-value weighting approach, leveraging structural insights to derive data-driven weights that prioritize the importance of different hypotheses. Our theoretical analysis demonstrates that LASLA asymptotically controls the false discovery rate (FDR) under independent or weakly dependent $p$-values, and achieves enhanced power in scenarios where the auxiliary data provides valuable side information. Simulation studies are conducted to evaluate LASLA's numerical performance, and its efficacy is further illustrated through two real-world applications.