P-Value Statistic Comes under Criticism

One such journal is JAMA Otolaryngology–Head and Neck Surgery. The publication now states in its editorial policies that the p-value is no longer a sufficient measurement for reporting results. An editorial published in 2016 states the journal’s new guidelines on results reporting (JAMA Otolaryngol Head Neck Surg. 2016;142:937–939).

The Laryngoscope and Laryngoscope Investigative Otolaryngology, both publications of the Triological Society, do not take the hard-lined stance that JAMA Otolaryngology–Head and Neck Surgery does. Authors can use any statistics they think are appropriate, and evaluating their relevance is part of the peer-review process. “In my opinion, the p-value still has some utility, and should be reported,” said Michael G. Stewart, MD, editor-in-chief of The Laryngoscope. “It is not sufficient however for making decisions about clinical significance, or meaningful clinical differences in outcomes or treatments, and other statistical measures. Confidence intervals, for example, are better for making those assessments.”

Although it isn’t clear how many other journals are or will be adopting similar changes to results reporting, there is clear interest in rethinking the p-value and how it is used in scientific research. According to Wasserstein, the ASA statement has generated widespread attention among scientific publications, and medical schools are emphasizing the principles in the statement to their students.

Cultural Shift: Embrace Uncertainty

For Wasserstein, the key challenge of shifting from an overreliance on the p-value to other ways of reporting results is cultural. “Naturally, all of us would love research that provides incontrovertible answers that generalize to every relevant situation, [but] actual research is noisier than that,” he said. “The challenge for researchers is to recognize and even embrace uncertainty, recognizing that evidence in difficult problems rarely sorts unambiguously into ‘something important is there’ and ‘nothing important is there.”

Clinicians, too, may feel a shift, but it may be best characterized by a renewed focus on the inherent ambiguity and uncertainty of medical care as it is practiced in the clinic and the ultimate importance of informed clinical judgment.

“Ultimately, clinicians must consider the study findings in the context of their own clinical practice and make a determination of whether they feel the data is clinically significant such that it impacts their current understanding of disease or the manner in which it should be treated,” said Dr. Soler.

Mary Beth Nierengarten is a freelance medical writer based in Minnesota.