The Parliamentary and Health Service Ombudsman (PHSO) use customer satisfaction data to verify their performance and ensure that the public can have confidence in their service. But what happens when the data is at odds with the accepted narrative? Is the data at fault or the narrative? And what exactly is measured in a single ‘customer satisfaction’ score?
On 2nd October 2019, PHSOtheFACTS members attended the third PHSO Open Meeting. In his recent blog post, Ombudsman Rob Behrens describes the purpose of the open meeting in the following way;
The purpose of the meeting is to explain in public what we do, listen to our service users, and learn from those engaged in front-line complaints resolution. Out of this we aim to build trust in PHSO as well as the organisations that we investigate. The event is a further illustration of PHSO’s transition into a more outward-facing, accessible, and transparent ombudsman service.
True to his word, engagement was facilitated by a Q & A session where previously submitted questions and those posed on the day via an on-line app called ‘Slido’ were put to the panel. We were pleased to learn that all the questions would be responded to on the PHSO website after the meeting, showing a willingness to answer the most difficult questions. It was therefore with some interest that I reviewed the answers in Mr Behrens recent blog post only to find that my question on customer satisfaction scores had been overlooked.
My question (seen at the top of this screenshot) is one I had previously put to Dr Chris Gill lecturer in Public Law at the University of Glasgow following the PACAC Peer Review scrutiny meeting. Having established that PHSO had been ‘an organisation in crisis’ following ‘failures in leadership’ Dr Gill and the peer review team used customer satisfaction data as one means of determining the scale of the recovery.
The panel was impressed by the systems in place to quality assure casework, in particular the use of customer data to verify performance against the Service Charter. Peer Review p31
Given that customer data is used to verify performance I asked Dr Gill why the customer satisfaction data was higher under Dame Julie Mellor’s leadership than it is now. It was a question he couldn’t answer. It can be seen from the chart that customer satisfaction was actually at its highest during the time that PHSO was suffering its organisational crisis. Given that ‘poor leadership’ was cited as the primary factor you would expect to see a steady decline in these scores from 2012, when Dame Julie Mellor took up the role to April 2017 when Mr Behrens became the new Ombudsman.
However, these figures do not appear to show an organisation which has gone through a crisis. In fact, the very opposite. The figures are remarkably stable, deviating from the average by just a few percentage points.
In the 2018/19 Annual Report we learn that the figures come from ‘snapshot’ data.
In 2018-19, our customer satisfaction survey captured feedback from 31% of the 1,837 people whose complaints we investigated and 5% of the 29,841 people whose complaints we closed at an earlier stage in our process. (p38)
Is the snapshot data adequate to accurately verify performance? How is the data analysed to produce a single ‘satisfaction’ score? Are complainants satisfied with the decision, the process or the service? Who is selected to give feedback and why? Unfortunately the Peer Review team did not unpack the detail on this issue and neither did it question the relatively high scores recorded at a time of ‘crisis’. There can be only two answers to this question:
Either PHSO did not suffer an organisational crisis or
the snapshot data does not accurately reflect the views of the service users.
Given that no external body monitors the Ombudsman’s investigation process, customer satisfaction data is the only external verification. Consequently, public trust relies upon the accuracy of this data and public trust will not be restored by side-stepping this question. Perhaps this is one for PACAC to ask at the next scrutiny session? Over to you Bernard.