We propose an Integrated Sensing and Communication (ISAC) algorithm that exploits the structure of a hierarchical codebook of beamforming vectors using a best-arm identification Multi-Armed Bandit (MAB) approach for initial alignment and tracking of a Mobile Entity (ME). The algorithm, called Dynamic Beam Zooming (DBZ), performs beam adjustments that mitigate the severe outages associated with wireless mmWave systems and allow for adaptive control of the parameters governing communications. We analyze the sample complexity of DBZ and use it to inform how the algorithm adapts to the nonstationary MAB statistics based on ME motion and Signal-to-Noise Ratio (SNR). We perform extensive simulations to validate the approach and demonstrate that DBZ is competitive against existing Bayesian algorithms, without requiring channel multipath or fading knowledge. In particular, DBZ outperforms other low-complexity algorithms in the low SNR regime. We also illustrate the efficacy of DBZ in standardized rural and urban scenarios using NYU Sim.