We develop a Bayesian approach called Bayesian projected calibration to address the problem of calibrating an imperfect computer model using observational data from a complex physical system. The calibration parameter and the physical system are parametrized in an identifiable fashion via $L_2$-projection. The physical process is assigned a Gaussian process prior, which naturally induces a prior distribution on the calibration parameter through the $L_2$-projection constraint. The calibration parameter is estimated through its posterior distribution, which provides a natural and non-asymptotic way for the uncertainty quantification. We provide a rigorous large sample justification for the proposed approach by establishing the asymptotic normality of the posterior of the calibration parameter with the efficient covariance matrix. In addition, two efficient computational algorithms based on stochastic approximation are designed with theoretical guarantees. Through extensive simulation studies and two real-world datasets analyses, we show that the Bayesian projected calibration can accurately estimate the calibration parameters, appropriately calibrate the computer models, and compare favorably to alternative approaches.