Natural language-based robotic navigation remains a challenging problem due to the human knowledge of navigation constraints, and destination is not directly compatible with the robot knowledge base. In this paper, we aim to translate natural destination commands into high-level robot navigation plans given a map of interest. We identify grammatically associated segments of destination description and recursively apply each of them to update a belief distribution of an area over the given map.We train a destination grounding model using a dataset of single-step belief update for precise, proximity, and directional modifier types. We demonstrate our method on real-world navigation task in an office consisting of 80 areas. Offline experimental results show that our method can directly extract goal destination from unheard, long, and composite text commands asked by humans. This enables users to specify their destination goals for the robot in general and natural form. Hardware experiment results also show that the designed model brings much convenience for specifying a navigation goal to a service robot.