We present an Extended Kalman Filter framework for system identification and control of a stochastic high-dimensional epidemic model. The scale and severity of the COVID-19 emergency have highlighted the need for accurate forecasts of the state of the pandemic at a high resolution. Mechanistic compartmental models are widely used to produce such forecasts and assist in the design of control and relief policies. Unfortunately, the scale and stochastic nature of many of these models often makes the estimation of their parameters difficult. With the goal of calibrating a high dimensional COVID-19 model using low-level mobility data, we introduce a method for tractable maximum likelihood estimation that combines tools from Bayesian inference with scalable optimization techniques from machine learning. The proposed approach uses automatic backward-differentiation to directly compute the gradient of the likelihood of COVID-19 incidence and death data. The likelihood of the observations is estimated recursively using an Extended Kalman Filter and can be easily optimized using gradient-based methods to compute maximum likelihood estimators. Our compartmental model is trained using GPS mobility data that measures the mobility patterns of millions of mobile phones across the United States. We show that, after calibrating against incidence and deaths data from the city of Philadelphia, our model is able to produce an accurate 30-day forecast of the evolution of the pandemic.