Behind the scenes at NICE: The justification for further delay to the ME/CFS guideline publication date

On March 29, the National Institute for Health and Care Excellence (NICE) announced that the expected publication date for the revised guideline for ME/CFS had been pushed back from 21 April 2021 to 18 August 2021 “because of the large number of comments received during consultation on the ME/CFS guideline, and the additional work needed to respond to them fully.”

Patients and stakeholders were left wondering why this long delay was announced just three weeks before the original publication date, and one week before an embargoed copy of the final guideline was due to be distributed to stakeholders.

More importantly, the lengthy delay prompted discussions and unease among the patient community regarding the justification given. Wouldn’t the large number of comments have been obvious soon after the consultation period on the draft guideline ended in late December 2020? Were there dark forces at play behind the scenes? Had Michael Sharpe handcuffed himself to the front entrance of the NICE offices in protest at the draft recommendations?

I decided to submit a Freedom Of Information (FOI) request to NICE to ascertain two things: (i) had NICE received consultation responses from stakeholders after the deadline, and if so were they discounted? and (ii) whether there were other reasons for the delay. The focus of this post is on the material provided to answer question (ii). Regarding (i), NICE did receive four late submissions, and these were discounted. The files from the FOI request (21MB) are available here. The files comprise PDFs of email threads, often with details (and some external emails) redacted, and four PDF files that were email attachments.

What I learned (My personal reading of the FOI material)

1. NICE knew early this year that it would likely need to delay the publication date. The delay was not implemented early on because they can only notify stakeholders of a Change in Publication Date (CPD) once. There were three CPD requests prior to the fourth and final request on March 26, which set a publication date of 18 August 2021.
2. The number of comments (over 4000) is not the only reason for the delay.
3. In emails on February 25 it is suggested that a month of extra time might be needed for possible extra work if the committee were to choose to conduct “any necessary reanalysis”. In response to this email, it is suggested that this delay seems reasonable because “for those studies that [stakeholders (SH)] claimed proportions of study population have [post-extertional malaise (PEM)], these aren’t reported in the original included studies, this information SH claimed are in other sources e.g. supplement papers, commentary papers, sister papers, interim reports, etc…NGC will have to retrieve all these additional sources of papers, check whether the SH’s claims are accurate, then do sensitivity analysis if the provided information is appropriate. Obtaining these additional publications, especially those are not open access (i.e. via British Library) will take a week or so.” (57_Redacted.pdf). An internal NICE email on 25 February headed “ME/CFS - addressing indirectness” mentions possible “additional work required to address the indirectness issue.” At the Committee meeting on March 5 2021, the panel discussed stakeholder comments regarding statements on indirectness in GRADE ratings. (GRADE is a method of assessing the certainty in evidence and the strength of recommendations in health care. GRADE was used in this guideline review and led to some key evidence being downgraded, prompting debate as to its suitability and implementation.) An internal NICE email sent during that meeting suggests a four-week delay for “the proposed approach to reanalysis” (6_Redacted.pdf).
4. An email on March 8 pertaining to the reanalysis reiterates the need for a delay in order that there is time “to find the papers and also hold a further committee meeting” (23_Redacted.pdf).
5. The final revised publication date was agreed in late March, shortly before NICE communicated the delay to stakeholders and updated the guideline webpages.
6. The wording of the email from NICE to stakeholders on March 29 was changed on the day, with the text in italics dropped from the statement: “Because of the large number of comments received during consultation on the ME/CFS guideline, and the need to undertake additional analysis to respond to them fully, the publication date has changed. The guideline will now publish on 18th August 2021” (25_Redacted.pdf). NICE has to stick to this date.

In summary, and as far as I can tell, it looks as though stakeholder comments on the draft have prompted a reanalysis of material to address issues with GRADE indirectness. You can read more about ‘indirectness’ within the context of GRADE here. My reading is that this probably concerns the sensitivity of recommendations to how representative patient cohorts were in studies, trials, or even patient surveys, particularly with regards to post-exertional malaise - though I could be wrong. Other patients/advocates who are familiar with GRADE and ‘indirectness’ may be able to piece together from this evidence what exactly this new reanalysis might constitute, but my feeling is that this process is primarily to address stakeholder comments and will not have a major impact on any recommendations.

Assuming that NICE did indeed proceed with this new (re)analysis, I don’t see why they couldn’t have been totally upfront with patients and other stakeholders as to the justification for the delay. Maybe they thought it would be met with even more concern.

I note that in a series of meetings in March, the committee was busy reviewing the recommendations in light of the stakeholder comments. If the minutes are accurate, out of the recommendations reviewed so far, the only change documented at this stage is to the CBT recommendations. This could be something as minor as changes to the phrasing of the recommendation, and I don’t think this has anything to do with the GRADE indirectness issue.