Ad hoc networks provide an on-demand, infrastructure-free means to communicate between soldiers in war zones, aid workers in disaster areas, or consumers in device-to-device (D2D) applications. Unfortunately, ad hoc networks are limited by interference due to nearby transmissions. Millimeter-wave (mmWave) devices offer several potential advantages for ad hoc networks including reduced interference due to directional antennas and building blockages, not to mention huge bandwidth channels for large data rates.. This paper uses a stochastic geometry approach to characterize the one-way and two-way signal-to-interference ratio distribution of a mmWave ad hoc network with directional antennas, random blockages, and ALOHA channel access. The interference-to-noise ratio shows that a fundamental limitation of an ad hoc network, interference, may still be an issue. The performance of mmWave ad hoc networks is bounded by the transmission capacity and area spectral efficiency. The results show that mmWave networks can support much higher densities and larger spectral efficiencies, even in the presence of blockage, compared with lower frequency communication for certain link distances. Due to the increased bandwidth, the rate coverage of mmWave can be much greater than lower frequency devices.