Predicting user influence in social networks is a critical problem, and hypergraphs, as a prevalent higher-order modeling approach, provide new perspectives for this task. However, the absence of explicit cascade or infection probability data makes it particularly challenging to infer influence in hypergraphs. To address this, we introduce HIP, a unified and model-independent framework for influence prediction without knowing the underlying spreading model. HIP fuses multi-dimensional centrality indicators with a temporally reinterpreted distance matrix to effectively represent node-level diffusion capacity in the absence of observable spreading. These representations are further processed through a multi-hop Hypergraph Neural Network (HNN) to capture complex higher-order structural dependencies, while temporal correlations are modeled using a hybrid module that combines Long Short-Term Memory (LSTM) networks and Neural Ordinary Differential Equations (Neural ODEs). Notably, HIP is inherently modular: substituting the standard HGNN with the advanced DPHGNN, and the LSTM with xLSTM, yields similarly strong performance, showcasing its architectural generality and robustness. Empirical evaluations across 14 real-world hypergraph datasets demonstrate that HIP consistently surpasses existing baselines in prediction accuracy, resilience, and identification of top influencers, all without relying on any diffusion trajectories or prior knowledge of the spreading model. These findings underline HIP's effectiveness and adaptability as a general-purpose solution for influence prediction in complex hypergraph environments.