X Close

NICE has created an ‘unholy mess’

Credit: Getty

August 26, 2021 - 4:06pm

I wrote a piece this week about ME/CFS, and the mystery of the new treatment guidelines being shelved. One of the things I found weirdest was that while NICE’s decision to delay publication had (as far as I could tell) been influenced by various clinicians’ groups — notably, some Royal Colleges, including those of psychiatrists, physicians, and paediatrics and child health — no one seemed to be willing to say why.

Since I wrote it, two people with some knowledge of the internal workings of both NICE and the Royal Colleges got in touch. They’re experts in relevant fields, and I know them both and trust them, so I thought I’d try to express the points they made.

First, they both said, NICE created a new definition of the disease when producing the new guidelines. The details aren’t hugely important, but because of the way the NICE evidence evaluation works (a system known as GRADE), the changed definition means that all evidence gathered under the old definition is automatically downgraded by one level, from “good” to “moderate” or from “moderate” to “low” etc.

According to my sources, this was the issue that the Royal Colleges had: that the new definition inevitably rendered all the evidence on GET low-quality. “It was quite well known it would have this effect,” said one, “so the colleges all wrote really quite detailed feedback.” None of that feedback has (yet) been published.

Essentially, the colleges said that this change would be bad for patients. My source made the point that while patients who have suffered for a long time with treatments that don’t work for them are likely to join campaign groups, those who see moderate improvement with GET and return to their lives probably won’t. So patients’ groups systematically tend to end up run by those patients with intractable conditions. I tended to conflate “patients’ groups” with “patients” in the piece, and perhaps it’s not as simple as that.

My sources also wanted to defend the PACE trial somewhat. Yes, they said, it’s true that it was unblinded and had subjective outcomes, meaning that it was open to bias. But given the difficulties, they say, it was probably as good as it could be: ME/CFS can only currently be diagnosed on subjective grounds, so subjective measures were necessary.

PACE has been accused of “outcome switching” — that is, changing what it was looking for mid-stream in order to make its results look better. I’ve even had people tell me it was “fraudulent”. I think that’s unfair: independent reviews say that changes to the outcome measures were reasonable and transparent. But it’s a huge ongoing row — see here; it appears that the changed outcomes did mean that; on one secondary measure, some patients were recorded as improving despite having got worse, although it didn’t affect the primary outcomes.

I should add that I somewhat misrepresented the PACE trial. I said it was CBT/GET vs standard medical care, where in fact there were four arms: standard medical care, CBT, GET, and “adaptive pacing therapy”: that is, planning and limiting your activity according to the restrictions of your disease. Pacing was the ME/CFS patients’ groups preferred method, but it showed no better outcome than standard care. I don’t think it materially changes any of my conclusions, but it is interesting that any bias in favour of CBT/GET didn’t show for pacing therapy. Also, it’s probably relevant that GET didn’t seem to have any worse side-effects than the other therapies, at least in this trial.

My understanding now is that the Royal Colleges were nervous that they were essentially being forbidden from offering GET, even though clinicians found that it was helpful for many patients. They may have also thought that the evidence supporting that ban was inadequate for the reasons discussed above. They suggested that the NICE GRADE system was badly designed for complex problems like ME/CFS. And they put all that in some long feedback, which, they say, was not then reflected in the final document.

And none of this feedback was made public. I am not endorsing any of what was said above, although I profoundly wish someone from NICE or the RCPCH or the various groups I contacted had talked to me, so I could have included it in the piece. But I do think this is the sort of thing that needs to be said out loud, so we can all assess it: as I said in my conclusion, NICE should have published the evidence, or — if the evidence wouldn’t make sense without the guidelines — some sort of explanation of what’s gone on. Somehow, the NICE process has got badly tangled up here. “It’s an unholy mess,” says one. “The colleges don’t want to come out publicly and say NICE’s committee screwed up.”


Tom Chivers is a science writer. His second book, How to Read Numbers, is out now.

TomChivers

Join the discussion


Join like minded readers that support our journalism by becoming a paid subscriber


To join the discussion in the comments, become a paid subscriber.

Join like minded readers that support our journalism, read unlimited articles and enjoy other subscriber-only benefits.

Subscribe
Subscribe
Notify of
guest

12 Comments
Most Voted
Newest Oldest
Inline Feedbacks
View all comments
Andrea X
Andrea X
2 years ago

It would be good to introduce the very many acronyms present here with their meaning otherwise I just can’t follow what is being said.
From memory:
NICE
GET
PACE
(Do they name things to fit the acronym, I wonder)
MS/CFS
And others I can’t remember.

Last edited 2 years ago by Andrea X
M War
M War
2 years ago
Reply to  Andrea X

National Institute of Health and Care Excellence (NICE)
Graded Exercise Therapy (GET)
Cognitive Behavioural Therapy (CBT)
“Pacing, graded Activity, and Cognitive behaviour therapy; a randomised Evaluation”) (PACE)
Myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS)
It’s actually ME/CFS, not MS/CFS.
A great lecture by Dr. Brian Hughes on why it’s the medical scandal of the century.
https://youtu.be/SY6ewZxJHV8

El Indio
El Indio
2 years ago

Since I wrote it, two people with some knowledge of the internal workings of…

According to my sources, this was…

I tended to conflate … with … in the piece, and perhaps it’s not as simple as that…

I should add that I somewhat misrepresented…

My understanding now is that…

Those are sentences you seldom see on ‘news’ websites. Revisiting a topic with reporting of new input, corrections to the conclusions and above all a sense of giving the reader the ability to make their own mind up, not just telling them what to think
Not every article on this site is like that, but the ratio is far higher than anywhere else, which is why I had to take out a sub recently.

Last edited 2 years ago by El Indio
M War
M War
2 years ago

From Dr. Brian Hughes, Ph.D
“- Studies relying on self-report require ‘blinding’
– The Pace trial relied heavily on self-report
– The PACE Trial did not employ blinding
“For a student we would say this is an example of a bad study.” – Dr. Brian Hughes
PACE Trial- “The greatest medical scandal of this century”
https://youtu.be/SY6ewZxJHV8

Sue Julians
Sue Julians
2 years ago
Reply to  M War

Disappointed with Tom’s follow up. I found it unconvincing.
The same argument is used for many therapies, and rightly derided.

M War
M War
2 years ago
Reply to  Sue Julians

I think the follow up reads more like a gossip column, than something written grounded in facts and scientific evidence.

Edward De Beukelaer
Edward De Beukelaer
2 years ago

As long as we shall be putting patients in boxes (diagnoses) based on anatomopathological thinking only, these stories will continue to happen. A patient is not only their illness, there is more to a patient than just the illness, hence one treatment may work for one patient and not the other. Time to be brave and accept that the anatomopathological approach to medicine is good at saving patients in acute situation and stopping people from dying but extremely poor at helting patients who have chronic issues. Time to move to multidisciplinary individualised medicine.

Gorgia Verolini-Wright
Gorgia Verolini-Wright
2 years ago

Never forget that Blair & Co created NICE (the single most inappropriate name for a health Nearly Govt Org) purely to shield ministers from taking decisions which would inevitably be controversial. The first time I came across them was when they banned the NHS from prescribing the only drug then available to mitigate the effects of MS in newly diagnosed patients, because they had noted that ONLY 2% of all MS patients would benefit. In other words, it was too costly – the facts were easily established that the 2% equalled 100% of patients recently diagnosed. These QUANGOS have (or are given) agendas which are not fit for purpose – i.e. there was absolutely nothing in that case which provided – or even tried to provide – any excellence at all. It was simply a cost saving measure which the politicians of the time could excuse themselves from having imposed on the most vulnerable patients in society – by which I clearly mean that they ONLY created this expensive monster simply to “hide behind” politically sensitive decisions. What other purpose do any of these bodies actually have & what “good” do any of them actually provide to the citizenry which was not already provided by the civil service?? In my view, it is all about “ruling” the people as easily as possible – excellence has nothing at all to do with it.

Last edited 2 years ago by Gorgia Verolini-Wright
sarah rutherford
sarah rutherford
2 years ago

Thanks you for writing about this. I was wondering a few days ago whether the condition of ME would be revisited given the more politically charged Long Covid issue but I didn’t know this was being done already. I personally am a big fan of the Lightning Process. I did it myself ..for another problem,,, because I met a couple of people who had been completely and I mean completely cured of severe ME during this three morning course. It’s NLP based and you have to believe in the connection between mind and body. I rather hope that today most people do….

Damon De Ionno
Damon De Ionno
2 years ago

I really appreciate the author issuing an update on the previous article in response to new information.
Extremely refreshing in the current media landscape.
This is why I paid to subscribe to Unherd and why I recommend it to others.

Geoffrey Wilson
Geoffrey Wilson
2 years ago

Interesting article, prompting me to read the earlier one and the comments. I have a lot of sympathy with the professionals trying to do their best in a context of angry and vociferous patient pressure groups. The modern world encourages us to get angry about our problems, which is a pity on so many levels. If the technicalities of the NICE process, including the change in definition point automatically downgrading previously respectable papers supporting GET, means that doctors will lose the only treatment they now have (even if it is only sometimes helpful) I can understand and support the Colleges’ dismay. Good luck to them and their patients.

M War
M War
2 years ago

GET/CBT is not based on evidence. It has a letter signed by a 100 Scientists calling for its retraction because it has so many flaws in it. Among the signatories, is Dr. Ron W Davis, Director of the Stanford Genome Centre. Garbage in. Garbage out. This is another Wakefield like mess that Richard Horton doesn’t have the balls to put out. The PACE Trial is taught at Ivy League Universities on “How not to conduct a clinical trial.” https://www.virology.ws/2015/10/21/trial-by-error-i/
Today, Mayo Clinic Proceedings ( the #13 Medical Journal) published it’s new ME/CFS Guidance based on evidence.
Patient groups have been treated by harmful non-evidenced based quacks for decades.
Patient groups want scientific evidence based care and patient centred care for our illness. You know like all the other illnesses. HIV/AIDS patients were angry in the 80’s too when they were mistreated. Good people had no problem with that, and they won’t when they understand that ME/CFS patients have been treated poorly based on pseudoscientific nonsense from powerfully connected people who care nothing about evidence or ethics.
https://www.mayoclinicproceedings.org/article/S0025-6196(21)00513-9/fulltext
Here are the two pertinent sections.
” Post-exertional Malaise and Energy Metabolism Impairment
In both healthy and sick people, physical exercise improves fatigue, sleep, pain, cognition, and mood.46,  47,  48,  49 In contrast, patients with ME/CFS experience PEM, a distinctive exacerbation of the patient’s set of symptoms and a further reduction in functioning after previously tolerated physical, cognitive, orthostatic, emotional, or sensory stressors. Multiple studies using both patient-reported and physiologic outcome measures have confirmed these accounts.1,50,  51,  52
In the past, some physicians and scientists speculated that these exertional limitations were due to physical deconditioning or an irrational fear of activity.53 While chronically inactive people are likely to be deconditioned, deconditioning does not explain the symptoms of ME/CFS. Instead, evidence suggests that problems generating and using the main energy molecule, adenosine triphosphate (ATP), may be a fundamental driver of ME/CFS.54
For example, when sedentary but healthy people or people affected by a number of other chronic illnesses are asked to exercise to their maximal ability on 2 consecutive days, energy test results do not change significantly from one day to the next. They may not use oxygen as efficiently as healthy, physically fit people, but their energy efficiency remains the same on repeated testing.55,  56,  57 In contrast, in ME/CFS, the ability to generate energy deteriorates on a repeated test the second day.55,57,  58,  59,  60 For instance, the work rate at ventilatory threshold can drop significantly, with 1 study reporting a drop of up to 55%.61
Other studies have reported high levels of lactate62,63 or increased acidosis64 in the blood, cerebrospinal fluid, and muscles. This could be due to increased production or decreased elimination. If aerobic metabolism is impaired, cells switch to anaerobic metabolic pathways instead, which produce more lactic acid but 18 times less ATP per glucose molecule.65 Repeated exercise improves lactic acid disposal in healthy people and other conditions but not in ME/CFS.64,66,67 Moreover, compared with patients moderately affected with ME/CFS, severely affected patients exhibited impairment in the glycolytic system as well.68 These changes may explain why patients have difficulty with tasks they tolerated before illness and with sustaining activities. Damage to more than 1 energy generation system may account for why severely affected patients are often so limited. Tomas and Newton69 and Rutherford et al70 have provided comprehensive reviews of these metabolic issues.
Exertion is also associated with changes in brain function and the immune system. Using functional magnetic resonance imaging, Cook et al51 found that altered brain activity accompanied post-exertional symptom exacerbation and impaired cognitive function. Maes et al71 found that PEM is associated with increased levels of interleukin-1, and Nijs et al72 revealed increased complement split products, oxidative stress, and gene expression of interleukin-10. Increased levels of immune system molecules in the brain, such as interleukin-1 and interleukin-10, can cause symptoms such as fatigue, pain, influenza-like feelings, and cognitive impairment. These objective changes correspond with and may contribute to the patient experiences of PEM. ”
Outdated Standard of Care
“In the past, CBT and GET were studied and recommended for ME/CFS on the basis of the disease theory that “the symptoms and disability of CFS/ME are perpetuated predominantly by unhelpful illness beliefs (fears) and coping behaviors (avoidance [of activity]),” leading to considerable deconditioning.53,108 However, GET and CBT studies have been widely criticized for their methodology, inadequate tracking of harms, and a disease theory that conflicts with the evidence of multisystem biologic impairment.4,108,  109,  110
The largest of these studies is the 2011 PACE (Pacing, graded Activity, and Cognitive behavior therapy; a randomised Evaluation) trial. PACE reported that these therapies were safe and resulted in recovery for 22% of participants and improvement for 60% to 61%.111,112 However, outcome measures were modified midtrial without a clear rationale.113 When the data were reanalyzed with the original protocol, improvement decreased by a factor of 3 and recovery rates decreased to 7% for CBT and 4% for GET, not significantly different from controls.113 The US Agency for Healthcare Research and Quality reported that many of these studies used definitions that could have included participants with other conditions and found little or no evidence of efficacy once these studies were excluded from the analysis.114 Finally, contradicting safety claims, 54% to 74% of patients have reported experiencing harms after GET.4
Because of these concerns, the US Centers for Disease Control and Prevention and health agencies in some countries have since removed recommendations for CBT and GET.115 Other nations are in the process of updating their guidance, with one stating that GET should not be offered as a treatment.116
Tom Chivers is listening to the words of the lobby group who have the most to benefit from the unscientific non-evidence based 2007 guidelines to continue. He is clearly not listening to scientific evidence that has so far gone “unheard” in the UK, nor to the ME/CFS patients who do not have the connections, wealth, and power that the PACE Trial authors do.
But both scientific evidence and scientific principles will win out in the end. With or without you.

Last edited 2 years ago by M War