The fact that Millimeter Wave (mmWave) communication needs to be directional is usually perceived as a challenge; in this paper we argue that it enables efficient secret key sharing that are unconditionally secure from passive eavesdroppers, by building on packet erasures. We showcase the potential of our approach in two setups: mmWave-based WiFi networks and vehicle platooning. We show that in the first case, we can establish a few hundred secret bits with minimal changes to standard communication protocol; while in both cases, with the right choice of parameters, we can potentially establish keys in the order of tenths of Mbps. These first results are based on some simplifying assumptions, yet we believe they give incentives to further explore such techniques.