We show that, for any Leray solution $u$ to the $3$D Navier--Stokes equations with $u_0\in L^2$, the associated deterministic and stochastic Lagrangian trajectories are unique for Lebesgue a.e. initial condition. Additionally, if $u_0\in H^{1/2}$, then pathwise uniqueness is established for the stochastic Lagrangian trajectories starting from every initial condition. The result sharpens and extends the original one by Robinson and Sadowski (Nonlinearity 2009) and is based on rather different techniques. A key role is played by a newly established asymmetric Lusin--Lipschitz property of Leray solutions $u$, in the framework of (random) Regular Lagrangian flows.