To what extent can news media help in providing more credible information about science? This is the core challenge for the Science Evidence Indicator (SEI) project, a collaboration between the Danish popular news website videnskab.dk and the authors of this paper. Looking specifically at medical science news, we aim to provide a transparent assessment of the scientific sources behind a story. This entails identifying some of the criteria that scientists use to assess research, and making it accessible and understandable for readers. We address the following research question: How can we communicate the quality of scientific publications in health science to a non-expert audience? Our goal is to make the assessments understandable for the youngest part of the website's target audience: high school students from age 16 and upwards.