The extensive use of smartphones and wearable devices has facilitated many useful applications. For example, with Global Positioning System (GPS)-equipped smart and wearable devices, many applications can gather, process, and share rich metadata, such as geolocation, trajectories, elevation, and time. For example, fitness applications, such as Strava and Runkeeper, utilize information for activity tracking, and have recently witnessed a boom in popularity. Those trackers have their own web platforms, and allow users to share activities on such platforms, or even with other social network platforms. To preserve privacy of users while allowing sharing, those platforms allow users to disclose partial information, such as the elevation profile for an activity, which supposedly will not leak the location trajectory. In this work we examine the extent to which publicly available elevation profiles can be used to predict the location trajectory of users. To tackle this problem, we devise three threat settings under which the city, borough, or even a route can be predicted. Those threat settings define the amount of information available to the adversary to launch the prediction attacks. Establishing that simple features of elevation profiles, e.g., spectral features, are insufficient, we devise both natural language processing (NLP)-inspired text-like representation and computer vision-inspired image-like representation of elevation profiles, and we convert the problem at hand into text and image classification problem. We use both traditional machine learning- and deep learning-based techniques, and achieve a prediction success rate ranging from 59.59% to 95.83%. The findings are alarming, and highlight that sharing information such as elevation profile may have significant privacy risks.