Under the Hood

Is Mindfulness Meditation Overhyped? Scientists May Be Exaggerating How Good It Is For Us

Stone pile
A recent review finds that research on whether mindfulness works is literally too good to be true. Pixabay, Public Domain

The science behind mindfulness techniques like meditation and yoga is getting overhyped by scientists themselves, suggests a recent study published earlier this April in PLOS ONE.

The mostly Canadian authors conducted a review to trump all reviews of the available research on mindfulness: Not only did they analyze 124 published randomized and controlled trials they also went back and evaluated 36 earlier reviews, specifically checking to see if these past efforts had crossed their t’s and dotted their i’s. In total, 88 percent of the trials reported at least one significant positive result, a suspiciously high percentage since the researchers calculated that only 53 percent of the trials should have turned out positive if mindfulness techniques were as effective as the sort of proven therapies someone would receive for depression, which they’re likely not. As for the 36 reviews, they found that nearly none had properly accounted for exaggerated results due to bias. Only three trials were brave enough to report negative results without any wishy-washy caveats.

“The proportion of mindfulness-based therapy trials with statistically significant results may overstate what would occur in practice,” the authors wrote.

One of the problems with mindfulness research, according to the authors, is that many studies are small, meaning that results are more likely to be a statistical fluke. Indeed, 47 percent of the trials they reviewed had fewer than 50 test subjects. Another issue comes with defining success itself: Try as they might, the authors couldn’t find a consistent measurable outcome, such as change in depression symptoms, among most of the trials. That makes it harder to pin down what exactly mindfulness does for our mental well-being. It also makes it much easier for scientists to cherry-pick any positive changes they saw in their subjects during the study and declare it a victory for mindfulness while neglecting to discuss any negative changes they saw.

The latter bias can be reduced by having scientists preemptively enroll their study in a publicly available registry such as ClinicalTrials.gov, where they can detail what they’re hoping to test and look for. Unfortunately, when the authors of the review looked through these registries, none of the 21 trials they found did so adequately. They also found evidence that many scientists aren’t publishing their research at all when the findings turn out negative, another example of reporting bias.

“The burden of registering a trial does not add substantively to the overall burden of designing, funding, conducting, and reporting a trial, and there are no real barriers to doing this,” the authors wrote.

For the sake of fairness, overhyping isn’t something unique to mindfulness research; researchers have found a similar gloss-over effect across every field of science — though it’s particularly commonplace in psychology. Last September, a review in PLOS ONE concluded that psychotherapy is less effective for major depression than it seems because negative studies have gone unpublished; the same team had previously discovered the same was true of pharmaceutical treatments.

These gaps aren’t typically nefarious — human error, unconscious bias, and the career pressure to publish flashy findings are often the driving factors behind overhyped research. But there are steps the scientific world can and should take to protect themselves from overhype. The authors of the mindfulness review recommend better training for trial researchers, larger-sized studies, and a concentrated push from academic journals to only publish studies that are properly preregistered.

Despite their findings, though, the authors aren’t dismissing mindfulness as a valuable treatment option, simply advocating a cautious skepticism for the time being.

“I have no doubt that mindfulness helps a lot of people,” senior author Dr. Brett Thombs told Nature. “I’m not against mindfulness. I think that we need to have honestly and completely reported evidence to figure out for whom it works and how much.”

Source: Coronado-Montoya S, Levis A, Kwakkenbos L, et al. Reporting of Positive Results in Randomized Controlled Trials of Mindfulness-Based Mental Health Interventions. PLOS ONE. 2016.