The benefits of mindfulness receive a lot of press (e.g., see the Huffington Post ). Mindfulness boosters frequently cite scientific studies to support the case for mindfulness as a kind of cure-all for the ills of the modern age. Given that mindfulness meditation involves the near-constant control of attentional processes and ongoing mental distancing through “observing” thoughts, labeling mental activity as “just thoughts” and gently redirecting attention away from thoughts, it makes a lot of sense that certain neuropsychological tendencies would be found in meditators. Given a worldview that values loving kindness and calm nonreactivity, it makes sense that mindfulness practitioners would report less stress and show fewer biomarkers for stress. It makes sense that mindfulness would be associated with greater well-being and happiness. Given hundreds or thousands hours of practice directing and redirecting attention, it makes sense that neural efficiency and connectivity patterns would be altered. The brain, body and personality all change with experience. If you spend hours and hours regulating cognitive, emotional and physiological processes in specific ways, your brain, body and personality will change in specific ways.
Questions remain regarding the mechanisms of change and how large and consistent these effects are. In books, blogs and the popular press one often statements that “researchers have found” or “studies show” without information on the quality or size of the studies involved or the robustness of the findings. When I check out the actual research, more often than not the researchers acknowledge the tentativeness of their conclusions and the need for replication. More often than not, the study design was not a randomized controlled trial and if even there was a control group, there was not a suitable comparison treatment condition. More often than not, the researchers did not appear to control for the placebo effect or factors common to most interventions (“common factors”) A few examples:
Take the study Prevention of Relapse/Recurrence in Major Depression by Mindfulness-Based Cognitive Therapy by Teasdale, Segal, Williams and others. This study has been frequently cited in academic papers and used by mindfulness advocates as strong evidence of the benefits of mindfulness. For instance, here’s how Jon Kabat-Zinn summarizes the study:
“…people with a prior history of three of more episodes of major depression taking the MBCT [Mindfulness-Based Cognitive Therapy] program relapsed at half the rate of the control group, which only received routine health care from their doctor…This was a staggering result…” (Full Catastrophe Living, Kindle p. 7322)
Now for some context. This particular study had no active comparison therapy. The control group received “treatment as usual” (aka routine health care). The MBCT group actually had a higher rate of relapse for participants who had two-or-fewer prior depressive episodes, not quite statistically significant but trending that way (p=>.10). The benefit for MBCT (for participants with 3+ prior episodes) was seen with a few as 4 treatment sessions (out of 8 possible) but the authors do not let us know if additional sessions (up to 8) increased benefit. We also have no idea what the actual ingredients of change are. Without an active comparison group that matches MBCT in factors common to all efficacious treatments, we don’t know if anything specific to MBCT made a difference in participant outcomes.
(Quick word about “common factors”: these include things like therapeutic alliance, empathy, goal consensus/collaboration, “buy-in”, positive regard/affirmation, and congruence/genuineness. Common factors are thought to exert much more influence over therapy outcomes than factors specific to individual therapies – for more on common factors, see Laska, Gurman and Wampold 2014.)
Other types of therapies have also been associated with reduced relapse in chronic depressives such as Maintenance Cognitive-Behavioral Therapy and Behavioral Activation Therapy. So when we are told the results of the MBCT study are “staggering”, I’m thinking: promising, yes – staggering, hardly. Mindfulness-based cognitive therapy clearly has some value; for one thing, it provides practical tools to help reduce stress and regulate unruly thoughts and emotions. It probably does help with unproductive rumination. But are mindfulness meditation and mindfulness-based therapies that much better than what’s already out there? Hard to say – since the quality of the research often leaves much to be desired.
Unless I want to spend the next decade on this project, I won’t be going into a lot of detail about each study that addresses the benefits of mindfulness. Let’s just look at a couple meta-analyses. One, “Mindfulness-based therapy: A comprehensive meta-analysis “(2013), concludes that mindfulness-based therapies are “an effective treatment for a variety of psychological problems”, but the authors also note that the moderate effectiveness of MBT “did not differ from traditional CBT [Cognitive-Behavioral Therapy] or behavioral therapies … or pharmacological treatments.”
The other meta-analysis was “The effect of mindfulness-based therapy on anxiety and depression: A meta-analytic review” (2010), which analyzed 39 studies (out of 727 originally identified as possible candidates for review). The authors found that mindfulness-based treatments were moderately effective for anxiety and depression, with stronger effects for individuals with anxiety and mood disorders. But their meta-analysis included many non-controlled studies, so how can we interpret these results?
Looking more closely at the 39 studies, 23 had no control or comparison group, 16 included a control or comparison group, of which 8 were waitlist controls, 3 were treatment-as-usual (TAU), and 5 actually had an active comparison treatment. So that’s 5 out of 39 MBT studies with a decent control group. But wait: of the 5 studies that were described as having “active controls”, two were “education programs” and two were types of art therapy. Education programs and art therapy are insufficient comparison treatments because they do not match the main intervention in common factors of efficacious treatments or placebo effects. (Note: I have designed such comparison interventions, so know a bit whereof I speak). Only one of the 5 studies listed as having an active control condition could be called an empirically supported “real” intervention – and that was cognitive-behavior group therapy, a condition with a grand total of 18 participants, representing just 1.5% of the 1,140 participants covered in the meta-analysis.
The authors of the 2010 meta-analysis actually criticize an earlier meta-analysis on the effect of mindfulness-based treatments partly because the authors of the earlier meta-analysis only reviewed controlled studies - and the other meta-analysis concluded that MBT does not have reliable effects on anxiety and depression. To quote: “Our study suggests that this conclusion was premature and unsubstantiated. The authors included only controlled studies, thereby excluding a substantial portion of the MBT research.”
Well, yeah, that is a legitimate problem. I’d recommend more high-quality controlled studies to address it. Then do another meta-analysis.
The problem with a lot of research on mindfulness is the same problem that plagues a lot of psychotherapy research: experimenter bias, which can taint even controlled studies. James Coyne puts this point beautifully in Salvaging Psychotherapy Research: a Manifesto:
“The typical RCT [Randomized Controlled Trial] is a small, methodologically flawed study conducted by investigators with strong allegiances to one of the treatments being evaluated. Which treatment is preferred by investigators is a better predictor of the outcome of the trial than the specific treatment being evaluated…Overall, meta-analyses too heavily depend on underpowered, flawed studies conducted by investigators with strong allegiances to a particular treatment or to finding that psychotherapy is in general efficacious. When controls are introduced for risk of bias or investigator allegiance, affects greatly diminish or even disappear.”
So, where does that leave us? With the need to do more, better research on mindfulness-based treatments. In the meantime, it’s probably safe to say that mindfulness practice and mindfulness-based treatments probably are helpful in some ways, for some people – but a lot of questions remain unanswered. To the degree that mindfulness advocates present evidence about the wonderful effects of mindfulness as unequivocal and/or uncontested (much less “staggering”), they are exaggerating and overstating their case.
Note: This post is also in Observing Mindfulness under the title “Mindfulness and the Ideological Square: Emphasize Our good things – Part II”
Reference: Jon Kabat-Zinn Full Catastrophe Living: Using the Wisdom of Your Body and Mind to Face Stress, Pain, and Illness, Kindle Version, Revised Edition 2013; Bantam Books, New York