In this paper, we propose a novel and effective approach to optimizing the load balancing in a millimeter wave (mmWave) cellular heterogeneous network (HetNet) with a macro-tier and a micro-tier. The unique characteristics of mmWave transmission are incorporated into the network by adopting the Poisson point process (PPP) for base station (BS) location, the line-of-sight (LoS) ball model for mmWave links, the sectored antenna model for key antenna array characteristics, and Nakagami-$m$ fading for wireless channels. To reduce the load of macro-tier BSs, we consider a bias factor $A_{s}$ in the network for offloading user equipments (UEs) to micro-tier BSs. For this network, we first analyze the loads of macro- and micro-tier BSs. Then we derive a new expression for the rate coverage probability of the network, based on which the optimal $A_{s}$ maximizing the rate coverage probability is found. Through numerical results, we demonstrate the correctness of our analysis and the validity of the optimal $A_{s}$. Importantly, the optimal $A_{s}$ can bring a profound improvement in the rate coverage probability relative to a fixed $A_{s}$. Furthermore, we evaluate the impact of various network parameters, e.g., the densities and the beamwidths of BSs, on the rate coverage probability and the optimal $A_{s}$, offering valuable guidelines into practical mmWave HetNet design.