Trajectory planning involves generating a series of space points to be followed in the near future. However, due to the complex and uncertain nature of the driving environment, it is impractical for autonomous vehicles~(AVs) to exhaustively design planning rules for optimizing future trajectories. To address this issue, we propose a local map representation method called Velocity Field. This approach provides heading and velocity priors for trajectory planning tasks, simplifying the planning process in complex urban driving. The heading and velocity priors can be learned from demonstrations of human drivers using our proposed loss. Additionally, we developed an iterative sampling-based planner to train and compare the differences between local map representations. We investigated local map representation forms for planning performance on a real-world dataset. Compared to learned rasterized cost maps, our method demonstrated greater reliability and computational efficiency.