Lancet Letter Exchange on Claimed Success of “Persistent Physical Symptoms” Trial Despite Clinically Insignificant Findings

By David Tuller, DrPH

Last year, The Lancet published a paper from Christopher Burton and colleagues called “Effectiveness of a symptom-clinic intervention delivered by general practitioners with an extended role for people with multiple and persistent physical symptoms in England: the Multiple Symptoms Study 3 pragmatic, multicentre, parallel-group, individually randomised controlled trial.” Per the norm for research on psycho-behavioral interventions for conditions that get dumped into the wastebasket category of “persistent physical symptoms,” the trial was unblinded and relied solely on subjective outcomes for its claims of “effectiveness.”

This study design is known for generating an unknown amount of bias. As repeatedly happens in such trials, the reported benefits on the primary outcome–in this case the mean of a health-related quality of life measure called the PHQ-15–were statistically significant but fell below the threshold designated as the minimal clinically important difference (MCID). The MCID is obviously a statistical construct, it is meant to approximate the level at which an improvement has practical or real-world impact on a patient’s experience of whatever it is.

Claiming effectiveness or success based on clinically insignificant findings is one of the strategies often deployed by investigators in Professor Burton’s field. It is disturbing that peer reviewers for major journals allow these claims to go unchallenged. The same can be said for the editorial team.

My friend and colleague Joan Crawford, a counseling psychologist, and I wrote a letter to The Lancet about this issue. It has just been published, along with a response from the investigators. (Thanks to Joan for spearheading this and shepherding it through the always irritating back-and-forth with the publisher.) The rebuttal is non-responsive to the concerns raised. It is always impossible to know to what extent the results in an unblinded trial relying solely on subjective measures are a measure of bias rather than genuine response to the intervention. That’s the point. It is unknowable. So any “positive” results cannot be taken at face value.

Beyond that, the mean results for the primary outcome in this trial fell below the MCID threshold. By definition, the reported change is meaningless from a clinical perspective at a population level. (That doesn’t mean none of the participants experience clinically significant improvements. Of course some proportion of them do.) In their vacuous response, the authors don’t mention their MCID problem. Instead, they suggest we are overlooking the “explanatory power of recent symptom science”–even though that purported “power” produced these embarrassing findings.

I have posted the published exchange below.

**********

Response to treatment in the Multiple Symptoms Study 3 trial
Joan Crawford, David Tuller

Christopher Burton and colleagues1 conducted an unblinded trial of a consultative intervention for 354 people with persistent physical symptoms but no identifiable “organic” cause. Patient Health Questionnaire-15 (PHQ-15) score, which is a brief, subjective, self-administered screen of severity of somatic complaints, was the primary outcome. This kind of trial design can be expected to produce modest positive outcomes, via expectation bias alone.2 No real-life, objective assessment of functioning was conducted.

The intervention involved up to four sessions with a general practitioner, offering patients “rational explanations” for their symptoms in the absence of known, falsifiable causes and helping patients to develop strategies for managing symptoms. Although the “rational explanations” are not described, they presumably did not include “organic” diseases or conditions.

Scores on the PHQ-15 range from 0 to 30. Recruited patients had scores from 10 to 20, indicating moderate severity. At the end of the study, the adjusted between-group difference of –1·82 did not reach the minimal clinically important difference of 2·3.3 The change is well within what would be expected from bias alone. The results therefore confirmed that the intervention was of limited, if any, practical benefit. However, the paper presents the findings as though they had shown the opposite.

Instead of the “rational explanations” promoted by the investigators, more straightforward and honest explanations—for example, we do not know what is causing symptoms—could easily have produced similar results. Patients desire resolution of symptoms and recovery of functions. The trial did not achieve this result. Post-intervention, participants continue to have an impaired quality of life and poor experiences of health care.

DT holds an academic position at the Center for Global Public Health at UC Berkeley, which is largely supported by crowdfunded donations directly to the university, many of them from patients with myalgic encephalomyelitis or chronic fatigue syndrome and related conditions that fall under the heading of persistent physical symptoms. JSC declares no competing interests.

References

1. Burton, C ∙ Mooney, C ∙ Sutton, L ∙ et al. Effectiveness of a symptom-clinic intervention delivered by general practitioners with an extended role for people with multiple and persistent physical symptoms in England: the Multiple Symptoms Study 3 pragmatic, multicentre, parallel-group, individually randomised controlled trial. Lancet. 2024; 403:2619-2629

2. Edwards, J. PACE team response shows a disregard for the principles of science. J Health Psychol. 2017; 22:1155-1158

3. Toussaint, A ∙ Kroenke, K ∙ Baye, F ∙ et al. Comparing the Patient Health Questionnaire-15 and the Somatic Symptom Scale-8 as measures of somatic symptom burden. J Psychosom Res. 2017; 101:44-50

**********

Response to treatment in the Multiple Symptoms Study 3 trial – Authors’ reply
Christopher Burton, Vincent Deary, Cara Mooney, Laura Sutton, Cindy Cooper

We thank Joan S Crawford and David Tuller for their interest in our Article.1 Persistent physical symptoms are heterogeneous and, by definition, do not tend to go away easily. In the absence of perfect knowledge about their pathophysiology, we believe that interventions to help people with multiple persistent physical symptoms should apply the knowledge that we currently have.2

Pragmatic interventions should have pragmatic evaluations (finding out whether something can be useful) rather than explanatory ones (using an effect to prove a mechanistic hypothesis).3 Multiple Symptoms Study 3 was a pragmatic trial of a complex, pragmatic intervention.1

The study evaluated outcomes shortly after treatment, at 6 months and 12 months. Although outcomes at the end of treatment might reflect a feeling of being heard, it is unlikely that this is the mechanism explaining the statistically significant difference between conditions at the 12-month follow-up point. Rather, we should look to the qualitative process evaluation (an expected part of a complex intervention trial4) to understand and interpret the findings.

Although feeling heard was important to patients, making sense of symptoms using current science and being offered a choice of actions based on this understanding were also valued by patients.5

Encouraging health professionals to proclaim “we do not know what is causing symptoms” in response to persistent physical symptoms, as Crawford and Tuller recommend, underestimates the explanatory power of recent symptom science and is likely to perpetuate the frustration and hopelessness of both clinicians and patients, which is all too common in this field.

CB has received publisher royalties and speaker honoraria in relation to persistent physical symptoms. All other authors declare no competing interests.

References

1. Burton, C ∙ Mooney, C ∙ Sutton, L ∙ et al. Effectiveness of a symptom-clinic intervention delivered by general practitioners with an extended role for people with multiple and persistent physical symptoms in England: the Multiple Symptoms Study 3 pragmatic, multicentre, parallel-group, individually randomised controlled trial. Lancet. 2024; 403:2619-2629

2. Löwe, B ∙ Toussaint, A ∙ Rosmalen, JGM ∙ et al. Persistent physical symptoms: definition, genesis, and management. Lancet. 2024; 403:2649-2662

3. Loudon, K ∙ Treweek, S ∙ Sullivan, F ∙ et al. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015; 350, h2147

4. Skivington, K ∙ Matthews, L ∙ Simpson, SA ∙ et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ. 2021; 374, n2061

5. Sanders, T ∙ Fryer, K ∙ Greco, M ∙ et al. Explanation for symptoms and biographical repair in a clinic for persistent physical symptoms. SSM Qual Res Health. 2024; 5, 100438

(View the original post at virology.ws)