Millimetre wave (mmWave) beam tracking is a challenging task because tracking algorithms are required to provide consistent high accuracy with low probability of loss of track and minimal overhead. To meet these requirements, we propose in this paper a new analog beam tracking framework namely Adaptive Tracking with Stochastic Control (ATSC). Under this framework, beam direction updates are made using a novel mechanism based on measurements taken from only two beam directions perturbed from the current data beam. To achieve high tracking accuracy and reliability, we provide a systematic approach to jointly optimise the algorithm parameters. The complete framework includes a method for adapting the tracking rate together with a criterion for realignment (perceived loss of track). ATSC adapts the amount of tracking overhead that matches well to the mobility level, without incurring frequent loss of track, as verified by an extensive set of experiments under both representative statistical channel models as well as realistic urban scenarios simulated by ray-tracing software. In particular, numerical results show that ATSC can track dominant channel directions with high accuracy for vehicles moving at 72 km/hour in complicated urban scenarios, with an overhead of less than 1\%.