National Guardian: Measuring Up?

By Dr Minh Alexander NHS whistleblower and former NHS consultant psychiatrist, 25 September 2017

 

The National Guardian has published her first tranche of data from NHS trusts on staff contacts with Freedom to Speak Up Guardians, for the period 1 April to 30 June 2017:

National Guardian data 20170831 FINAL Q1 PUBLISHED TABLE- v2

There is little discussion in her accompanying report about data quality:

National Guardian report on whistleblowing data Q1 2017:18

This is despite a past admission by the National Guardian that there was great variability between trusts in how data was being gathered:

https://minhalexander.files.wordpress.com/2017/02/hh-meeting-records-23-01-2017-and-2-02-2017.pdf

The lack of discussion of data quality is all the more significant given that fact that a whopping one third of trusts  (n=88) failed to return any data at all.

Four trusts rated as ‘Outstanding’ by the CQC were amongst the 88 trust that failed to return any data:

Birmingham Women’s and Children’s NHS Foundation Trust East London NHS Foundation Trust Salford Royal NHS Foundation Trust Newcastle upon Tyne Hospitals NHS Foundation Trust

 

This is the full list of the 88 trusts that failed to return any data:

88 Trusts that returned no data to the National Guardian Q1 2017:18

The National Guardian does not account in her report for why such a large number of trusts failed to return data nor does she make proposals for improving future response rates.

Another key issue is that the system rests entirely on self reporting by trusts.

There is no indication that the National Guardian intends to pursue any form of verification.

Indeed, she was reluctant in a past discussion to consider external quality control:

From records of a telephone meeting with the National Guardian 23 January 2017:

“MA What data are local guardians collating and has it been agreed with your office? Is it standardised?

HH “What” has been agreed with us and is standardised “How” – no. “How” is very much what works for the organisation. So long as information held separately from main databases. We haven’t been prescriptive.

MA How do you know data from local Guardians isn’t fiddled or flawed in some other way?

HH Ultimately it will reflect in the staff survey.

MA Have you got a means of quality control to check quality of local guardians’ data?

HH We’re working on trust….

MA So no checks?

HH We don’t have access to their information systems. Are you suggesting we do so? Is that appropriate?

MA You have the remit for picking up local failures including by local guardians, and safeguarding against local failure.

HH We haven’t received any information about problems with local guardians”

 

The National Guardian has now admitted that 17 trusts reported that no staff made any contact with Freedom To Speak Up Guardians:

17 Trusts reported that there were no contacts with their Speak Up Guardians

Birmingham and Solihull Mental Health NHS FT Blackpool Teaching Hospitals NHS FT Calderdale and Huddersfield NHS FT Cambridgeshire Community Services NHS Trust Central Manchester University Hospitals NHS FT East Kent Hospitals University NHS FT Homerton University Hospital NHS FT London Ambulance Service NHS Trust Norfolk and Norwich University Hospitals NHS FT North Staffordshire Combined Healthcare NHS Trust Royal Free London NHS FT Royal Surrey County Hospital NHS FT Royal United Hospitals Bath NHS FT Somerset Partnership NHS FT Southend University Hospital NHS FT The Queen Elizabeth Hospital King’s Lynn NHS FT Wrightington, Wigan and Leigh NHS FT

 

This is a tad awkward as the National Guardian herself previously maintained that numbers of contacts were an indication of staff confidence:

From records of a meeting  with the National Guardian on 2 February 2017

“MA But how will you measure and prove that it’s working well?

HH 1) Numbers of staff going to the Speak Up Guardians – that’s a measure of confidence”

 

Notably, Blackpool Teaching Hospitals NHS Foundation Trust was amongst the trusts that reported zero contacts with their Speak Up Guardians.

The Speak Up Guardian for this trust, a deputy medical director, was much feted and featured in the National Guardian’s recent publicity material, including a particularly embarassing promotional video.

Conversely, troubled Colchester Hospital University NHS Trust reported a high number of contacts (30), but it also had the highest number of reports of detriment (14).

Colchester has latterly been run by the former of CEO of Croydon Health NHS Trust, which recently lost against NHS whistleblower Dr Kevin Beatt in the Court of Appeal.

Other trusts which reported high numbers of contacts with Speak Up Guardians included the following:

East Lancashire and Mid Yorkshire Hospitals are notorious amongst whistleblowers.

Out of a total of 1,124 cases were raised in 144 trusts, 192 (17%) were raised anonymously.

It is unclear how cases were raised anonymously. But anonymous reporting is not an unequivocal vote of confidence in the Speak Up Guardians.

It is also alarming that there were concerns about lack of places where Guardians could meet staff confidentially. This is a basic issue that should have been sorted out by now, and reveals a lack of sensitivity and insight into the huge risks that staff take when whistleblowing.

Lastly, the National Guardian maintains that there was “an overwhelmingly positive response” from staff who had made disclosures to Speak Up Guardians, and she states that 87% said they would speak up again.

No details are given in her report about how this staff feedback data was gathered or the completeness of the data that she cites.

In any case, one third of trusts are missing from the National Guardian’s statistics and this is a potential source of bias. For example, Trusts with poorer results may have been less willing to return data.

As a doctor, the National Guardian should appreciate the importance of properly caveating any limitations in data.

Previously, she informed me that Speak Up Guardians would seek staff feedback at 3 months post contact. However, it seems unlikely that the staff experience data she cites was subsequently gathered on this basis because her report was published on or before 5 September. This would not have allowed for  a full 3 month follow up of all contacts for the relevant period.

She also previously admitted that there was no uniformity in how Speak Up Guardians were measuring staff experience of their services:

 

From records of a meeting with the National Guardian 2 February 2017:

“MA How are Speak Up Guardians measuring staff experience?

HH It varies, Different trusts need different things

MA What are the different ways in which Speak Up Guardians are measuring staff experience?

HH Working in partnership at local level. Pulse surveys as well as the annual staff survey.

MA What are the pulse survey questions?

HH We haven’t received the outputs of those yet.

MA But what questions are being asked?

HH We haven’t asked specifically”

 

However, this lack of standardisation is not acknowledged in her latest report, other than by implication in a comment that the numbers of staff contacts cannot be used to compare between trusts.

Some staff reported worries about repercussions.

Staff in 85 of the 1,124 (7.5%) cases raised by Speak Up Guardians reported detriment.

The period of follow up so far is not long enough to fully assess whether all the staff who raised concerns will suffer reprisal. Reprisal can occur long after disclosure behind closed doors, once no one is looking.

The National Guardian should also have acknowledged this limitation in her data.

In short, the National Guardian’s claim of overwhelming positive staff feedback is presently not clearly substantiated.

For transparency and accountability, the National Guardian should:

• Clearly describe the methodology by which staff feedback is obtained
• Provide trust level data on staff feedback, including response rates
• Publish the surveys used by Speak Up Guardians to gather staff feedback.

I have written to ask if she will do so.

 

 

Email 24 September 2017 to the National Guardian:

BY EMAIL

Dr Henrietta Hughes

National Freedom To Speak Up Guardian

Care Quality Commission

24 September 2017

Dear Dr Hughes,

 

Re: Speak Up data for all trusts April – July 2017

I see that you have published broad findings on staff feedback as part of this data:

http://www.cqc.org.uk/national-guardians-office/content/speaking-data

Is it possible in future data releases to provide the following data:

– The methodology by which staff feedback is obtained by Speak Up Guardians

– Trust level data on staff feedback, including response rates

– Copies of the surveys used by local Speak Up Guardians to measure staff feedback ?

Many thanks,

Minh

Dr Minh Alexander

 

 

RELATED ITEMS

Letter to PAC

Letter to Public Accounts Committee 11 Sep 2017 Re-Review of whistleblowing

CQC denies denial

https://minhalexander.com/2017/09/16/newsflash-cqc-denies-denial/

SSOTP: Robert Francis’ exemplar trust has feet of clay and Jeremy Hunt’s safety claims are unevidenced

https://minhalexander.com/2017/05/28/ssotp-robert-francis-exemplar-trust-has-feet-of-clay-and-jeremy-hunts-safety-claims-are-un-evidenced/