Accurate precise positioning at millimeter wave frequencies is possible due to the large available bandwidth that permits precise on-the-fly time of flight measurements using conventional air interface standards. In addition, narrow antenna beamwidths may be used to determine the angles of arrival and departure of the multipath components between the base station and mobile users. By combining accurate temporal and angular information of multipath components with a 3-D map of the environment (that may be built by each user or downloaded a-priori), robust localization is possible, even in non-line-of-sight environments. In this work, we develop an accurate 3-D ray tracer for an indoor office environment and demonstrate how the fusion of angle of departure and time of flight information in concert with a 3-D map of a typical large office environment provides a mean accuracy of 12.6 cm in line-of-sight and 16.3 cm in non-line-of-sight, over 100 receiver distances ranging from 1.5 m to 24.5 m using a single base station. We show how increasing the number of base stations improves the average non-line-of-sight position location accuracy to 5.5 cm at 21 locations with a maximum propagation distance of 24.5 m.