We propose a new method called localized conformal prediction, where we can perform conformal inference using only a local region around a new test sample to construct its confidence interval. Localized conformal inference is a natural extension to conformal inference. It generalizes the method of conformal prediction to the case where we can break the data exchangeability, so as to give the test sample a special role. To our knowledge, this is the first work that introduces such a localization to the framework of conformal prediction. We prove that our proposal can also have assumption-free and finite sample coverage guarantees, and we compare the behaviors of localized conformal prediction and conformal prediction in simulations.