This letter proposes a one-shot algorithm for feature-distributed kernel PCA. Our algorithm is inspired by the dual relationship between sample-distributed and feature-distributed scenario. This interesting relationship makes it possible to establish distributed kernel PCA for feature-distributed cases from ideas in distributed PCA in sample-distributed scenario. In theoretical part, we analyze the approximation error for both linear and RBF kernels. The result suggests that when eigenvalues decay fast, the proposed algorithm gives high quality results with low communication cost. This result is also verified by numerical experiments, showing the effectiveness of our algorithm in practice.