Millimeter-wave (mmWave) communication is a promising solution for achieving high data rate and low latency in 5G wireless cellular networks. Since directional beamforming and antenna arrays are exploited in the mmWave networks, accurate angle-of-arrival (AOA) information can be obtained and utilized for localization purposes. The performance of a localization system is typically assessed by the Cramer-Rao lower bound (CRLB) evaluated based on fixed node locations. However, this strategy only produces a fixed value for the CRLB specific to the scenario of interest. To allow randomly distributed nodes, stochastic geometry has been proposed to study the CRLB for time-of-arrival-based localization. To the best of our knowledge, this methodology has not yet been investigated for AOA-based localization. In this work, we are motivated to consider the mmWave cellular network and derive the CRLB for AOA-based localization and its distribution using stochastic geometry. We analyze how the CRLB is affected by the node locations' spatial distribution, including the target and participating base stations. To apply the CRLB on a network setting with random node locations, we propose an accurate approximation of the CRLB using the L/4-th value of ordered distances where L is the number of participating base stations. Furthermore, we derive the localizability of the mmWave network, which is the probability that a target is localizable, and examine how the network parameters influence the localization performance. These findings provide us deep insight into optimum network design that meets specified localization requirements.