The Kalman filter (KF) is a widely-used algorithm for tracking dynamic systems that are captured by state space (SS) models. The need to fully describe a SS model limits its applicability under complex settings, e.g., when tracking based on visual data, and the processing of high-dimensional signals often induces notable latency. These challenges can be treated by mapping the measurements into latent features obeying some postulated closed-form SS model, and applying the KF in the latent space. However, the validity of this approximated SS model may constitute a limiting factor. In this work, we study tracking from high-dimensional measurements under complex settings using a hybrid model-based/data-driven approach. By gradually tackling the challenges in handling the observations model and the task, we develop Latent-KalmanNet, which implements tracking from high-dimensional measurements by leveraging data to jointly learn the KF along with the latent space mapping. Latent-KalmanNet combines a learned encoder with data-driven tracking in the latent space using the recently proposed-KalmanNet, while identifying the ability of each of these trainable modules to assist its counterpart via providing a suitable prior (by KalmanNet) and by learning a latent representation that facilitates data-aided tracking (by the encoder). Our empirical results demonstrate that the proposed Latent-KalmanNet achieves improved accuracy and run-time performance over both model-based and data-driven techniques by learning a surrogate latent representation that most facilitates tracking, while operating with limited complexity and latency.