In this paper, we present a new approach to improving the relevance and reliability of medical IR, which builds upon the concept of Level of Evidence (LoE). LoE framework categorizes medical publications into 7 distinct levels based on the underlying empirical evidence. Despite LoE framework's relevance in medical research and evidence-based practice, only few medical publications explicitly state their LoE. Therefore, we develop a classification model for automatically assigning LoE to medical publications, which successfully classifies over 26 million documents in MEDLINE database into LoE classes. The subsequent retrieval experiments on TREC PM datasets show substantial improvements in retrieval relevance, when LoE is used as a search filter.