Importance sampling (IS) is a widely used simulation method for estimating rare event probabilities. In IS, the relative variance of an estimator is the most common measure of estimator accuracy, and the focus of existing literature is on constructing an importance measure under which the relative variance of the estimator grows sub-exponentially as the parameter increases. In practice, constructing such an estimator is not easy. In this work, we study the behavior of IS estimators under an importance measure which is not necessarily optimal using large deviations theory. This provides new insights into asymptotic efficiency of IS estimators and the required sample size. Based on the study, we also propose new diagnostics of IS for rare event simulation.