Reading and Assessing the Clinical Research Literature

The key results should be presented in either tabular or graphic form, highlighting important points in the text. Once again, measures of central tendency, variance, and range should be provided and the type of statistical test (t-test, chi square, ANOVA, Mann-Whitney U test, etc.) used to determine statistical significance specified. Usually 0.05 is stipulated as the de facto level of significance; bear in mind, however, that statistical significance may not necessarily denote clinical significance. To paraphrase Gertrude Stein, A difference, to be a difference, must make a difference. Significance values can be influenced by sample size (the larger the sample size, the greater the likelihood of finding a significant effect), the variance (the larger the variance, the smaller the likelihood of finding a significant effect), and the method of collecting the data.

Discussion and Conclusions

Having examined the results as described and presented graphically, you will have formed your own opinion as to what conclusions can be drawn. Does the author reach the same conclusions-or are they overstated, or even erroneous? Insufficient follow-up time is an important source of error in assessing treatment effects. Have alternative explanations for the results been considered? Has the central hypothesis been accepted or rejected by the results of the study? What is the clinical significance of this finding? Where should research go from this point?

On a more personal basis, the reader will want to ask whether the reported results can be applied to the patients in his or her own practice. Were the study patients similar to the practice patients in demographics and case mix? Were all clinically important outcomes, or those that would be important in the practice considered? Were the treatment benefits worth the likely costs and risks? The issues of validity and relevance are key in determining a paper’s clinical value to you and to the field.

The Bottom Line

Five simple questions will help you make a quick assessment of the quality of a paper:3

• Does the study have internal validity-was it designed so you can trust the findings?
• Does the study have external validity-was it designed so you can generalize the findings to your patients?
• Is the study important-what was the magnitude of the effect?
• Was the study reliable-if the study was repeated, is it likely that the same or similar results would be obtained?
• Was systematic bias avoided or minimized?

The time spent identifying high-quality papers, whether for teaching purposes, journal clubs, use as background for other papers or grant applications, or for application to clinical practice, will be rewarded by more efficient critical reading, more dependably relevant to your clinical interests-and perhaps will improve your own publications!

References

1. Guyatt G, Rennie D (eds). Users’ Guides to the Medical Literature. Chicago: AMA Press, 2002.z
   [Context Link]
2. Straus SE, Richardson WS, Glasziou P, Haynes RB. Evidence-based Medicine. How to Practice and Teach EBM (3rd ed). Edinburgh: Elsevier Churchill Livingstone, 2005.
   [Context Link]
3. Hurley Research Center. Critical appraisal of the medical literature. www.hurleymc.com/upload/docs/Medical%20Literature.ppt#256,1 .
   [Context Link]

©2008 The Triological Society