The matrix factorization (MF) technique has been widely adopted for solving the rating prediction problem in recommender systems. The MF technique utilizes the latent factor model to obtain static user preferences (user latent vectors) and item characteristics (item latent vectors) based on historical rating data. However, in the real world user preferences are not static but full of dynamics. Though there are several previous works that addressed this time varying issue of user preferences, it seems (to the best of our knowledge) that none of them is specifically designed for tracking concept drift in individual user preferences. Motivated by this, we develop a Temporal Matrix Factorization approach (TMF) for tracking concept drift in each individual user latent vector. There are two key innovative steps in our approach: (i) we develop a modified stochastic gradient descent method to learn an individual user latent vector at each time step, and (ii) by the Lasso regression we learn a linear model for the transition of the individual user latent vectors. We test our method on a synthetic dataset and several real datasets. In comparison with the original MF, our experimental results show that our temporal method is able to achieve lower root mean square errors (RMSE) for both the synthetic and real datasets. One interesting finding is that the performance gain in RMSE is mostly from those users who indeed have concept drift in their user latent vectors at the time of prediction. In particular, for the synthetic dataset and the Ciao dataset, there are quite a few users with that property and the performance gains for these two datasets are roughly 20% and 5%, respectively.