Citation metrics are widely used to assess academic impact but suffer from social biases, including institutional prestige and journal visibility. Here we introduce rXiv Semantic Impact (XSI), a novel framework that predicts research impact by analyzing how scientific semantic graphs evolve in underlying fabric of science. Rather than counting citations, XSI tracks the evolution of research concepts in the academic knowledge graph (KG). Starting with a construction of a comprehensive KG from 324K biomedical publications (2003-2025), we demonstrate that XSI can predict a paper's future semantic impact (SI) with remarkable accuracy ($R^2$ = 0.69) three years in advance. We leverage these predictions to develop an optimization framework for research portfolio selection that systematically outperforms random allocation. We propose SI as a complementary metric to citations and present XSI as a tool to guide funding and publishing decisions, enhancing research impact while mitigating risk.