Prior works have explored multi-armed bandit (MAB) algorithms for the selection of optimal beams for millimeter-wave (mmW) communications between base station and mobile users. However, when the number of beams is large, the existing MAB algorithms are characterized by long exploration times, resulting in poor overall communication throughput. In this work, we propose augmenting the upper confidence bound (UCB) based MAB with integrated sensing and communication (ISAC) to address this limitation. The premise of the work is that the radar and communication functionalities share the same field-of-view and that communication mobile users are detected by the radar as mobile targets. The radar information is used for significantly reducing the number of candidate beams for the UCB, resulting in an overall reduction in the exploration time. Further, the radar information is used to estimate the realignment time in quasi-stationary scenarios. We have realized the MAB and radar signal processing algorithms on the system on chip (SoC) via hardware-software co-design (HSCD) and fixed-point analysis. We demonstrate the significant gain in execution time using accelerators. The simulations consider complex propagation channels involving direct and multipath, with simple and extended radar targets in the presence of significant static clutter. The resulting experiments show that the proposed ISAC-based MAB achieves a 35% reduction in the overall exploration time and 1.4 factor higher throughput as compared to the conventional MAB that is based only on communications.