Directed last passage percolation models on the plane, where one studies the weight as well as the geometry of optimizing paths (called polymers) in a field of i.i.d. weights, are paradigm examples of models in the KPZ universality class. In this article, we consider the large deviation regime, i.e., when the polymer has a much smaller (lower tail) or larger (upper tail) weight than typical. Precise asymptotics of large deviation probabilities have been obtained in a handful of the so-called exactly solvable scenarios, including the Exponential (Johansson, '00) and Poissonian (Seppäläinen, '98 and Deuschel, Zeitouni, '99) cases. How the geometry of the optimizing paths change under such a large deviation event was considered in (Deuschel, Zeitouni, '99), where it was shown that the paths (from $(0,0)$ to $(n,n)$, say) remain concentrated around the straight line joining the end points in the upper tail large deviation regime, but the corresponding question in the lower tail was left open. We establish a contrasting behavior in the lower tail large deviation regime, showing that conditioned on the latter, in both the models, the optimizing paths are not concentrated around any deterministic curve. Our argument does not use any ingredient from integrable probability, and hence can be extended to other planar last passage percolation models under fairly mild conditions; and also to other non-integrable settings such as high dimensions.