The ever-anomalous CQC. Another soft-shoe shuffle around inconvenient data.

By Dr Minh Alexander NHS whistleblower and former consultant psychiatrist 18 November 2016

Summary

There is currently an unprecedented level of grave concern about the state of mental health services, which has today prompted nine past Health Secretaries to attack the government’s failures on mental health. The CQC annual community mental health survey – which measures what patients say about mental health services – was published earlier this week and presents further anomalies. It raises more questions about the validity and consistency of the CQC’s inspection ratings. The survey response rate is persistently low and the survey does not seem to distinguish well between trusts. Nor do CQC’s inspection ratings correspond reliably with survey results. Notwithstanding, the survey results support the concerns and questions by campaigners about why a particularly troubled trust, Norfolk and Suffolk NHS Foundation, has been released from special measures.

Background

Today, The Times has reported:

“Every health secretary from the past 20 years has condemned the “enduring injustice” faced by patients with mental illnesses and accused the government of failing to honour pledges to help them.

In an unprecedented intervention, the nine previous holders of the post say they are “alarmed and dismayed” that little has changed since the promise last year that the NHS would treat mental health on a par with physical problems. The last two chief executives of the health service join them in warning that “warm words” were yet to be backed by action to alleviate the suffering of families nationwide.”

https://minhalexander.files.wordpress.com/2016/11/times-nine-health-secretaries-attack-government-for-failing-mentally-ill.pdf

The CQC oversees a national community mental health survey every year, in which adult patients receiving specialist mental health care in the community are sent a questionnaire about their experience of care. [1]

Mental health trusts are expected to arrange the sampling, either through “approved contractors or in house”, which introduces potential variation and bias. Data has been excluded from the survey, both this year and in the past, due to sampling deemed as flawed.

It is a costly exercise as 850 questionnaires are sent out by each trust, and two subsequent rounds of reminders, to non-responders are also sent.

However, the typical response rate is low and falls short of a target response rate of 40%. This year the overall response rate was just 28%. The range was 18.1% to 33.9%.

The response rate has not varied greatly over the years, raising questions about whether effective action has been taken to elucidate why the rate is low, or to address this.

YEAR	CQC Community Mental Health Survey – rate of response
2014	29%
2015	29%
2016	28%

Source: annual community mental health survey reports

It would be useful to know if the most ill, most vulnerable or most dissatisfied patients are not responding to the survey. The CQC admits that it cannot answer this question and it implausibly claims that there is no way to find out:

“However, whether we do have non-response bias is difficult to assess, as we do not have any way of knowing how those people who did not respond would have answered.”

One hopes the survey in its current form and current execution is not hiding the full extent of problems.

Things are bad enough as it is. The annual surveys have repeatedly revealed serious problems with mental health care, as might be expected from chronic underfunding and the second class status of mental health services. A persistently serious finding is how dissatisfied patients are with crisis care. Crisis resolution and home treatment teams have been used less as a positive, safely resourced, genuine attempt to provide less restrictive care, but more as a means of gatekeeping dwindling beds, sometimes inappropriately so and with fatal results. [2]

The survey questions are scored out of ten (0 being worst and 10 being best). Survey questions grouped into ten groups, and each group is given its own overall rating of “better”, “about the same” or “worse” than other trusts based on the scores.

CQC insists that the statistical process means that these ratings are robust and accurate:

“The technique used to analyse these results allows us to identify which trusts we can confidently say performed ‘better’, ‘worse’ or ‘about the same’ when compared with other trusts.”[3]

Famous last words? It is clear from a bird’s eye view that the great majority of mental health trusts have ratings of “about the same”. This suggests that the survey and or CQC’s rating system may be rather blunt instruments, and are not very good at distinguishing between trusts:

https://minhalexander.files.wordpress.com/2016/11/cqc-ratings-community-mental-health-survey-scores-2015-161.xlsx

There are also conflicts with CQC’s inspection ratings. Northumberland Tyne and Wear NHS Foundation Trust (NTW) was recently pronounced “Outstanding” by CQC after an inspection led by Paul Lelliott CQC deputy chief inspector. [4] However, the community mental health survey data reportedly places it as “about the same” as other trusts, across all groups of questions. East London NHS Foundation trust (ELFT) was similarly declared “Outstanding’ [5], but is rated “about the same” as other trusts in nine out of ten of the groups of questions.

There are two noticeable exceptions in this sea of “about the sameness”. These are likely to be significant outliers particularly given the survey’s seeming tendency to lump trusts together. The two exceptions are:

2gether NHS Foundation Trust

According to patients, notwithstanding the questions over the accuracy and representativeness of the survey data, and validity of CQC’s ratings, this is the best mental health trust. It scored “better” on four out of 10 groups of questions. See Table 1.

Table 1. 2gether NHS Foundation Trust Community mental health survey 2016. Summarised results:

There are also other indications of above average performance by 2gether. For example, on the national staff survey. 2gether’s staff survey results are above average in almost all domains, more so than NTW:

https://minhalexander.files.wordpress.com/2016/11/2gether-and-ntw-summarised-staff-survey-results-juxtaposed.docx

However, the CQC has deemed that 2gether is not as good at NTW and ELFT – it was rated only ‘Good”, and not ‘Outstanding’. [6]

Norfolk and Suffolk NHS Foundation Trust

Patients have given this very troubled trust the thumbs down and it was rated “worse” than other trusts in five out of ten groups of questions. See Table 2.

Table 2. Norfolk and Suffolk NHS Foundation Trust Community mental health survey 2016. Summarised results:

Norfolk and Suffolk suffered dangerous cuts in staffing several years ago, with a round of much publicised and controversial redundancies. It suffered disproportionately harsher cuts than acute services in the same area. The CQC admitted in February 2015 that NSFT was ‘Inadequate’ and unsafe, and the trust was placed in special measures. Curiously, the CQC inspection report of February 2015 did not give any data on trust deaths. In fact, the only mention of deaths was this misleading statement:

“Every six months the Ministry of Justice publishes a summary of Schedule 5 recommendations (previously rule 43) which had been made by coroners with the intention of learning lessons from the cause of death and preventing further deaths. In the latest report covering the period from October 2012 to March 2013 there were no concerns regarding the trust raised by the coroner”[7]

CQC omitted to mention that there had been several coroners’ warnings after March 2013. [8]

NSFT deaths have been mounting and most recently the trust management has been accused of hiding risks to patients by failing to provide details of unexpected deaths in the trust annual report. [9]

CQC recently uprated NSFT from ‘Inadequate’ to ‘Requires Improvement’, after an inspection chaired by Paul Lelliott CQC Deputy Chief Inspector. [10] Very controversially, NSFT was also released from special measures. [11] Local campaigners hope to mount a legal challenge to the trust’s release from special measures, subject to raising sufficient funds for a judicial review.

The uprating of NSFT and its release from special measures was announced shortly before Alan Yates the improvement director previously installed at NSFT [12] – by Monitor – was recycled and appointed as interim chair at another huge embarrassment to the Minister, Southern Health NHS Foundation Trust. [13]

CQC et al may live to regret putting a PR sticking plaster on the severe safety problems at NSFT.

At a rate of almost four unexpected deaths a week according to campaigners, there is only so much one can do to hide the bodies.

“Is NSFT the mental health Mid Staffs?

….Since April 2012, at least 572 NSFT patients have died unexpectedly; the number has increased every year. Based upon the first five months of 2016-17, more than 200 NSFT patients will die unexpected deaths this year, which will take the total since April 2012 above 750. NSFT claims to take the deaths of those who rely on its services seriously, but then removes their number from the trust’s document of historical record. This is shameful.” [14]

4 thoughts on “The ever-anomalous CQC. Another soft-shoe shuffle around inconvenient data.”

Eileen Chubb says:

November 22, 2016 at 12:43 pm

Well done, Keep up the brilliant work. Eileen Chubb

LikeLike

minhalexander says:

November 22, 2016 at 12:44 pm

🙂 Thanks Eileen. Team effort. We plod on.

LikeLike

Myrtle says:

November 29, 2016 at 8:19 pm

Thank you for doing this.

I live in the area covered by Northumberland, Tyne and Wear NHS Foundation Trust, and was gobsmacked to learn of their “outstanding” rating.

One issue in particular I feel is important is to do with who fills in the questionnaires, as you have mentioned. My particular concern is that there seems to be no monitoring/evidence of who is actually able to access services. It is a sample only of those who have received services, and notably excludes those who perhaps should be receiving services but have fallen in the gap between primary and secondary care, due to ever increasing thresholds to secondary care caused by dwindling budgets. (I do not know what happens if people in this situation complete suicide, especially if the paper trail points to repeated attempts to seek help, eg. referrals from GP, but I suspect nothing much.)

I actually received a questionnaire myself a few years ago, although I am not sure if it was CQC one or just something the CMHT did. However the limited set questions in it meant I decided there was no point filling it in. I would have been able to tick “adequate” or “good” for all the things they asked whilst still having an extremely poor experience of the service, and ultimately not receiving any treatment (despite promises of therapy it never materialised). I believe there was a box for “additional comments” but if one has recorded everything they’ve asked as at least “adequate”, then one looks vindictive and “mad”(!!) to then complain in this section. In much the same way, the official complaints procedure does not allow for a “catalogue of errors” only very specific complaints…

Once again, thank you – it takes someone from the “inside” to speak out in mental health, as the rest of us can be considered unrealiable due to the very problems that caused us to seek help int the first place…

LikeLike

minhalexander says:

November 30, 2016 at 9:27 am

Thank you Myrtle. Very interesting feedback. Many thanks for taking the time.

LikeLike