Rank models play a key role in industrial recommender systems, advertising, and search engines. Existing works utilize semantic tags and user-item interaction behaviors, e.g., clicks, views, etc., to predict the user interest and the item hidden representation for estimating the user-item preference score. However, these behavior-tag-based models encounter great challenges and reduced effectiveness when user-item interaction activities are insufficient, which we called "the long-tail ranking problem". Existing rank models ignore this problem, but its common and important because any user or item can be long-tailed once they are not consistently active for a short period. In this paper, we propose a novel neighbor enhancement structure to help train the representation of the target user or item. It takes advantage of similar neighbors (static or dynamic similarity) with multi-level attention operations balancing the weights of different neighbors. Experiments on the well-known public dataset MovieLens 1M demonstrate the efficiency of the method over the baseline behavior-tag-based model with an absolute CTR AUC gain of 0.0259 on the long-tail user dataset.