Evidence-Based Practice for Nursing: Evaluating the Evidence

Evaluating Evidence: Questions to Ask When Reading a Research Article or Report

For guidance on the process of reading a research book or an article, look at Paul N. Edward's paper, How to Read a Book (2014). When reading an article, report, or other summary of a research study, there are two principle questions to keep in mind:

1. Is this relevant to my patient or the problem?

• Once you begin reading an article, you may find that the study population isn't representative of the patient or problem you are treating or addressing. Research abstracts alone do not always make this apparent.
• You may also find that while a study population or problem matches that of your patient, the study did not focus on an aspect of the problem you are interested in. E.g. You may find that a study looks at oral administration of an antibiotic before a surgical procedure, but doesn't address the timing of the administration of the antibiotic.
• The question of relevance is primary when assessing an article--if the article or report is not relevant, then the validity of the article won't matter (Slawson & Shaughnessy, 1997).

2. Is the evidence in this study valid?

• Validity is the extent to which the methods and conclusions of a study accurately reflect or represent the truth. Validity in a research article or report has two parts: 1) Internal validity--i.e. do the results of the study mean what they are presented as meaning? e.g. were bias and/or confounding factors present?; and 2) External validity--i.e. are the study results generalizable? e.g. can the results be applied outside of the study setting and population(s)?
• Determining validity can be a complex and nuanced task, but there are a few criteria and questions that can be used to assist in determining research validity. The set of questions, as well as an overview of levels of evidence, are below.

For a checklist that can help you evaluate a research article or report, use our checklist for Critically Evaluating a Research Article

How to Read a Paper--Assessing the Value of Medical Research

Evaluating the evidence from medical studies can be a complex process, involving an understanding of study methodologies, reliability and validity, as well as how these apply to specific study types. While this can seem daunting, in a series of articles by Trisha Greenhalgh from BMJ, the author introduces the methods of evaluating the evidence from medical studies, in language that is understandable even for non-experts. Although these articles date from 1997, the methods the author describes remain relevant. Use the links below to access the articles.

Not all published research is worth considering. This provides an outline of how to decide whether or not you should consider a research paper.

Greenhalgh, T. (1997b). How to read a paper. Getting your bearings (deciding what the paper is about). BMJ (Clinical Research Ed.), 315(7102), 243–246.

This article discusses how to assess the methodological validity of recent research, using five questions that should be addressed before applying recent research findings to your practice.

Greenhalgh, T. (1997a). Assessing the methodological quality of published papers. BMJ (Clinical Research Ed.), 315(7103), 305–308.

This article and the next present the basics for assessing the statistical validity of medical research. The two articles are intended for readers who struggle with statistics

Greenhalgh, T. (1997f). How to read a paper. Statistics for the non-statistician. I: Different types of data need different statistical tests. BMJ (Clinical Research Ed.), 315(7104), 364–366.

Greenhalgh, T. (1997). Education and debate. how to read a paper: Statistics for the non-statistician. II: "significant" relations and their pitfalls. BMJ: British Medical Journal (International Edition), 315(7105), 422-425. doi: 10.1136/bmj.315.7105.422

Greenhalgh, T. (1997d). How to read a paper. Papers that report drug trials. BMJ (Clinical Research Ed.), 315(7106), 480–483.

Greenhalgh, T. (1997c). How to read a paper. Papers that report diagnostic or screening tests. BMJ (Clinical Research Ed.), 315(7107), 540–543.

Greenhalgh, T. (1997e). How to read a paper. Papers that tell you what things cost (economic analyses). BMJ (Clinical Research Ed.), 315(7108), 596–599.

Greenhalgh, T. (1997i). Papers that summarise other papers (systematic reviews and meta-analyses). BMJ (Clinical Research Ed.), 315(7109), 672–675.

Greenhalgh, T., & Taylor, R. (1997). Papers that go beyond numbers (qualitative research). BMJ (Clinical Research Ed.), 315(7110), 740–743.

Levels of Evidence

In some journals, you will see a 'level of evidence' assigned to a research article. Levels of evidence are assigned to studies based on the methodological quality of their design, validity, and applicability to patient care. The combination of these attributes gives the level of evidence for a study. Many systems for assigning levels of evidence exist. A frequently used system in medicine is from the Oxford Center for Evidence-Based Medicine. In nursing, the system for assigning levels of evidence is often from Melnyk & Fineout-Overholt's 2011 book, Evidence-based Practice in Nursing and Healthcare: A Guide to Best Practice. The Levels of Evidence below are adapted from Melnyk & Fineout-Overholt's (2011) model.

Uses of Levels of Evidence: Levels of evidence from one or more studies provide the "grade (or strength) of recommendation" for a particular treatment, test, or practice. Levels of evidence are reported for studies published in some medical and nursing journals. Levels of Evidence are most visible in Practice Guidelines, where the level of evidence is used to indicate how strong a recommendation for a particular practice is. This allows health care professionals to quickly ascertain the weight or importance of the recommendation in any given guideline. In some cases, levels of evidence in guidelines are accompanied by a Strength of Recommendation.

About Levels of Evidence and the Hierarchy of Evidence: While Levels of Evidence correlate roughly with the hierarchy of evidence (discussed elsewhere on this page), levels of evidence don't always match the categories from the Hierarchy of Evidence, reflecting the fact that study design alone doesn't guarantee good evidence. For example, the systematic review or meta-analysis of randomized controlled trials (RCTs) are at the top of the evidence pyramid and are typically assigned the highest level of evidence, due to the fact that the study design reduces the probability of bias ( Melnyk , 2011), whereas the weakest level of evidence is the opinion from authorities and/or reports of expert committees. However, a systematic review may report very weak evidence for a particular practice and therefore the level of evidence behind a recommendation may be lower than the position of the study type on the Pyramid/Hierarchy of Evidence.

About Levels of Evidence and Strength of Recommendation: The fact that a study is located lower on the Hierarchy of Evidence does not necessarily mean that the strength of recommendation made from that and other studies is low--if evidence is consistent across studies on a topic and/or very compelling, strong recommendations can be made from evidence found in studies with lower levels of evidence, and study types located at the bottom of the Hierarchy of Evidence. In other words, strong recommendations can be made from lower levels of evidence.

For example: a case series observed in 1961 in which two physicians who noted a high incidence (approximately 20%) of children born with birth defects to mothers taking thalidomide resulted in very strong recommendations against the prescription and eventually, manufacture and marketing of thalidomide. In other words, as a result of the case series, a strong recommendation was made from a study that was in one of the lowest positions on the hierarchy of evidence.