P-Value Statistic Comes under Criticism

by Mary Beth Nierengarten •

Clinical medicine is anything but precise. To practice medicine well, clinicians need to recognize the many influences that can change, for example, a heart that is working well to one that is in failure. Environment, aging, social influences, and the many unknowns surrounding the workings of the mind on the body all shape this transition from health to disease.

In its essence, clinical medicine involves ambiguity, and practitioners are required to manage, and even be comfortable with, a degree of uncertainty when evaluating and treating a patient. And yet, the research on which practitioners rely to provide good information on how to diagnose and treat patients has, for many years, been based on a measurement that has given at least the illusion of certainty. Use of the p-value to signify that a finding is positive, particularly at an arbitrarily set threshold of <0.05, has governed what is considered “significant” and has strongly influenced which data is accepted for publication and how that data is interpreted for clinical decision making.

But all this is changing. Although many clinicians and researchers over the years have questioned our overreliance on the p-value as a measure that provides sufficient information on which to base the practice of good clinical medicine, a concerted effort to supplement the p-value with other measures more aptly applied to clinical medicine is rapidly evolving.

This evolution is changing the way in which research is conducted, the criteria used for publication, and, importantly, the emphasis on clinical relevance versus statistical significance when interpreting data.

Clinical Relevance

“Many disciplines have come to rely on bright-light thresholds (such as p<0.05) as a means of filtering what is scientifically meaningful from what is not,” said Ronald L. Wasserstein, executive director of the American Statistical Association (ASA), based in Alexandria, Va. “Such thresholds are simple to apply and have the appearance of objectivity. The results, unfortunately, are much less objective than they appear,” he added.

In 2016, the ASA released a statement outlining six principles that clarify the proper use and interpretation of the p-value (Am Stat. 2016;70:129–133; see “The Proper Use and Interpretation of the P-Value,”). What these principles highlight is the limitations of relying solely on the p-value to provide sufficient information for interpreting data.

For Timothy Smith, MD, MPH, director of the Oregon Sinus Center, chief of rhinology and sinus-skull base surgery, and director of clinical research in the department of otolaryngology–head and neck surgery at Oregon Health and Sciences University in Portland and a member of ENTtoday’s Editorial Advisory Board, the ASA statement reinforces what he thinks many clinicians have sensed for a long time. “I think clinicians generally are a bit wary of p-values and significant findings,” he said.