This letter considers the problem of end-to-end learning for joint optimization of transmitter precoding and receiver processing for mmWave downlink positioning. Considering a multiple-input single-output (MISO) scenario, we propose a novel autoencoder (AE) architecture to estimate user-equipment(UE) position with multiple base-stations (BSs) and demonstrate that end-to-end learning can match model-based design, both for angle of departure (AoD) and position estimation, under ideal conditions without model deficits and outperform it in the presence of hardware impairments.