The Parliamentary and Health Service Ombudsman (PHSO) use customer satisfaction data to verify their performance and ensure that the public can have confidence in their service. But what happens when the data is at odds with the accepted narrative? Is the data at fault or the narrative? And what exactly is measured in a single ‘customer satisfaction’ score?
On 2nd October 2019, PHSOtheFACTS members attended the third PHSO Open Meeting. In his recent blog post, Ombudsman Rob Behrens describes the purpose of the open meeting in the following way;
The purpose of the meeting is to explain in public what we do, listen to our service users, and learn from those engaged in front-line complaints resolution. Out of this we aim to build trust in PHSO as well as the organisations that we investigate. The event is a further illustration of PHSO’s transition into a more outward-facing, accessible, and transparent ombudsman service.
True to his word, engagement was facilitated by a Q & A session where previously submitted questions and those posed on the day via an on-line app called ‘Slido’ were put to the panel. We were pleased to learn that all the questions would be responded to on the PHSO website after the meeting, showing a willingness to answer the most difficult questions. It was therefore with some interest that I reviewed the answers in Mr Behrens recent blog post only to find that my question on customer satisfaction scores had been overlooked.
My question (seen at the top of this screenshot) is one I had previously put to Dr Chris Gill lecturer in Public Law at the University of Glasgow following the PACAC Peer Review scrutiny meeting. Having established that PHSO had been ‘an organisation in crisis’ following ‘failures in leadership’ Dr Gill and the peer review team used customer satisfaction data as one means of determining the scale of the recovery.
The panel was impressed by the systems in place to quality assure casework, in particular the use of customer data to verify performance against the Service Charter. Peer Review p31
Given that customer data is used to verify performance I asked Dr Gill why the customer satisfaction data was higher under Dame Julie Mellor’s leadership than it is now. It was a question he couldn’t answer. It can be seen from the chart that customer satisfaction was actually at its highest during the time that PHSO was suffering its organisational crisis. Given that ‘poor leadership’ was cited as the primary factor you would expect to see a steady decline in these scores from 2012, when Dame Julie Mellor took up the role to April 2017 when Mr Behrens became the new Ombudsman.
However, these figures do not appear to show an organisation which has gone through a crisis. In fact, the very opposite. The figures are remarkably stable, deviating from the average by just a few percentage points.
In the 2018/19 Annual Report we learn that the figures come from ‘snapshot’ data.
In 2018-19, our customer satisfaction survey captured feedback from 31% of the 1,837 people whose complaints we investigated and 5% of the 29,841 people whose complaints we closed at an earlier stage in our process. (p38)
Is the snapshot data adequate to accurately verify performance? How is the data analysed to produce a single ‘satisfaction’ score? Are complainants satisfied with the decision, the process or the service? Who is selected to give feedback and why? Unfortunately the Peer Review team did not unpack the detail on this issue and neither did it question the relatively high scores recorded at a time of ‘crisis’. There can be only two answers to this question:
Either PHSO did not suffer an organisational crisis or
the snapshot data does not accurately reflect the views of the service users.
Given that no external body monitors the Ombudsman’s investigation process, customer satisfaction data is the only external verification. Consequently, public trust relies upon the accuracy of this data and public trust will not be restored by side-stepping this question. Perhaps this is one for PACAC to ask at the next scrutiny session? Over to you Bernard.
phsothetruestory 
PHSO has disclosed its ‘Matrix of complainant survey questions’:
https://www.whatdotheyknow.com/request/869584/response/2062964/attach/3/Matrix%20of%20complainant%20survey%20questions.pdf?cookie_passthrough=1
It includes, for example:
‘I feel that the investigation has been thorough’
The possible responses include:
‘Strongly agree | Tend to agree | Neither agree nor disagree | Tend to disagree | Strongly disagree’
(This is asked regarding Investigation pre-decision/Investigation post-decision)
The raw data relating to some of the questions might shed more light than the information provided in:
‘Performance against our Service Charter 2021/22 Quarter 1 (April to June)’
https://www.ombudsman.org.uk/publications/performance-against-our-service-charter-202122-quarter-1-april-june/following-open-and-fair-process
LikeLike
Yes PHSO are the masters of spin.
LikeLike
PHSO 23/3/22
No negative feedback from stakeholders!
PHSO Stakeholder Surveys 2021
‘Stakeholders were asked how positively or negatively they felt about PHSO.
58% gave a positive response, 42% gave a neutral response but none gave a negative response.’ (p22)
Click to access PHSO_Stakeholder_Surveys_2021.pdf
Trustpilot gives a different impression:
https://www.whatdotheyknow.com/request/phsos_trustpilot_account#incoming-1485481
LikeLike
Stakeholders must be the ones let off the hook.
LikeLike
Looking at the latest PHSO_Stakeholder_Surveys_2021 (based on responses from only 4,000 public, and only 100 MPs) it very much seems to be a PHSO staff morale boosting effort since nothing independently traceable or even reliable or balanced as the full picture, is given. Then, words selected ambitiously to represent PHSO qualities like transparent, fair, impartial, even excellence have to be put in context with words for complainants such as ‘unhappy’; a universal PHSO term for all ‘complainers’ who raise serious concerns with PHSO service. The PHSO’s trick is not to measure it through having no equitable process to do that.
LikeLike
Rigged feedback?
Here is a comment I made on WDTK some time ago:
“There is evidence of the PHSO targeting people who have been complimentary:
‘Do you remember the names of the individuals from last night who said that they had a positive experience of PHSO? I am keen to ensure that we keep their details as further down the line PHSO may want to ask them if they would like to be involved in another activity going forward.’ (p.65)”
https://www.whatdotheyknow.com/request/positive_feedback_stored_on_meri#comment-62536
This remark was made by a ‘Grade E2 member of staff – the very top!
https://www.whatdotheyknow.com/request/remark_by_unidentifed_member_of#comment-86044
LikeLike
Their customer feedback survey has been rigged for years which is why it didn’t dip when PHSO supposedly lost their way under Dame Julie Mellor.
LikeLike
Maybe I forgot to add I’m just another dissatisfied customer who was in PHSO terminology “Not happy” that my son died having been failed and abused under UK Equality Acts, under NHS Charter, under “expectation” of best practice… but I was never asked my opinion of the outcome. Would they appreciate my actual opinion and feedback? NO is the answer, because “Feedback option” is barred if their victims ask them to explain abusive attitudes and unfounded conclusions. “Only nice positive feedback, please” . The service is duplicitous and a National scandal.
LikeLike
The only reason my complaint was partly upheld and presumably hailed as “Resolution” or ‘Success’ was that the NHS Trust bartered with PHSO to drop all relevant charges in favour of a failing which (a) was not the reason for complaint (b) was not relevant to cause of death and (c) was not NHS duty to provide – it being a Social Services failure of legal requirement. As usual the PHSO had taken each contributory factor, diluted it with their own opinions, played it down, dismissed my evidence as opinion, used unsupported NHS opinion (lies), THEN having washed their hands of any substantive failures stated “On balance, we could see not see failings….”.
All of this was in abuse of law, authority and capacity, and total shite; yet R Behrens personally aligned himself with this and failed to produce the review promised for 6 months following Oct 2017 – and still awaited. Needless to say I asked to see this “review” but was told there was nothing to see. I had already rejected Draft outcome (and hence the Final outcome 2016) due to this collaboration, and unsupported and unprofessional PHSO statements causing actual harm to me my family – a point Behrens dismissed then without consultation, and has dismissed ever since, whilst making ‘public arena’ gain of how great he was for the admitted failing quango. There is a pattern obvious.
I’m sorry Mr Behrens if this is the way you want to twist and distort facts and then write them off again with no investigation (similar to my PHSO Complaints Dec 2013 ongoing Sept 2017 ) something stinks bad. My son is one of many others affected who will never get to trouble the PHSO again. However my Proxy Complaints are Unresolved and I am still suffering the after effects of your corporate deceit and personal negligence in failing to be being candid and transparent. Your honour is severely in question.
LikeLike
The same thing that’s been going on for many years with the PHSO… because they are not monitored and no one appears to be able to call them to account for anything, they do what they please with the data, the statistics, the cases, and the ‘reviews’. In my experience, I did not even ask for a case to be opened nor was I aware of their procedures, but they used every manipulation possible to squeeze information out of me and then twist it into a report which of course, they finalised and concluded (wrongly) after I explicitly told them to cancel their investigation. Many letters and emails went back and forth, which served the purpose of exhausting me, and delaying any potential legal action by 3 years. They are completely corrupt and only serve to (a) block the individual and public right to complain and (b) keep themselves in a position of authority which they frequently abuse.
LikeLike
This is spot on James. Would you like to put your case story in our PHSO review?
LikeLike
The only consistency in these figures is that the more likely a complaint is to be upheld then the more likely the complainant is to be satisfied. That is hardly surprising! Strangely 14 % of people whose complaint was fully upheld in the most recent year were not satisfied and of course we are told that 47% of people whose complaint was not upheld were satisfied.
As to the point whether a complaint is upheld is split into three, then the question of satisfaction should be split as there are degrees of satisfaction. Some people may be satisfied with the way it was handled but not with the result and it is not clear where they would come. Are they put in with the 47% satisfied even when their complaint was not upheld? What about people who are satisfied with one or two aspects but on the whole dissatisfied? Where are they placed? Apart from the fact that these figures are remarkably (and almost unbelievably) consistent within a range of about 10% they are very vague and give very little detail as to how ‘satisfaction’ is defined.
LikeLiked by 1 person
Spot on!
LikeLike
Customer satisfaction questionnaires are designed and manipulated to get the answers they want. No one can be satisfied with their cases unresolved, they must think complainants are stupid. When the 4,000 plus cases of complaint by the 1950s born women regarding the Govt maladministration and failure on their Sate Pensions, are not investigated by PHSO, will the figures the following year show good customer satisfaction? Of course they will. Cases were banked up at the Independent Complaints Examiner (ICE) purposely and then closed down in an instant. The small proportion of only 6 ‘representative’ cases ( which is ridiculous as every case has its own unique characteristics) are still waiting for the Ombudsman to judge if they will be investigated. The excuse to delay again due to an Appeal by another pension group on different matters, is a disgrace. Typical of the Ombudsman system, to delay for months to decide whether to investigate or not and then more likely come out with the ‘it’s not in our remit’ phrase. How could there possibly be any customer satisfaction on a system so corrupt PHSO. These figures are a con.
LikeLike
Quite right Anne. When 92% of complaints are closed within 7 days how can the Ombudsman be getting satisfaction rates above 50%? These scores cannot be trusted.
LikeLike
It all boils down to the inane questions they ask. They don’t ask questions related to how they investigate, only whether they were efficient and polite and those sorts of questions. There’s no other explanation and this was how I was questioned on the one occasion I was contacted by their market research company. The questions didn’t at all fit what I wanted to say about PHSO. Regarding Behrens omitting questions asked by the public at the Open Meeting in October, you weren’t the only one. I sent a list of 6 questions to the dedicated email address of openmeeting@ombudsman.org.uk and not a single one was asked or put up on their website (https://www.ombudsman.org.uk/news-and-blog/events/annual-open-meeting-2019-qa), which allegedly lists the questions they didn’t get round to at the meeting. They are liars and cover-up merchants. Who they think they are fooling I don’t know. I’ve used the #PHSO hashtag on Twitter so you can find my Tweet and keep RTing it to show them up. Includes a snapshop from the email of the questions they ignored.
LikeLike
The organisational crisis was and possibly is the fact they they have been challenged.
LikeLike
Very true Peggy. Historic failure to lose an Ombudsman mid-term.
LikeLike
Reblogged this on | truthaholics.
LikeLike
Thanks for this excellent analysis. I have also been uneasy regarding the figures for some years.
We must ask, since the majority of complainants never have their complaints investigated by the PHSO, why the satisfaction rate is not a lot worse than it is. Most complain to make a difference so why should they remain satisfied when their complaint is not investigated? If these rejected cases have been wrongly directed to the PHSO, again why should they be satisfied by misinformation? When this majority realise that it is considered they have nothing to contribute from their experience and there is nothing to be improved, why should they be satisfied?
Surveys have to be carefully constructed to try to faithfully represent the complainant view. How they are carried out, how the sampling is done and what questions are asked need to be laid open.
Why the customer satisfaction data appears higher under Dame Julie Mellor’s leadership than it is now remains unexplained. For a crude, personal analysis, we can turn to the October Open Meeting in Manchester, Peter Tyndall (Irish Ombudsman for Ireland and President of the International Ombudsman Institute) gave his view that though the PHSO was on its knees, it is ‘walking again’ but he tellingly added, it has ‘yet to fly’. Considering the interaction and mutual support between academics and Ombudsmen over the years, one wonders what is holding the PHSO back and just why Peter Tyndall thinks it has ‘yet to fly’?
If the PHSO aim is ‘to explain in public what we do,’ and to ‘build trust in PHSO’, then for a ‘transparent ombudsman service’, one wonders why your question hasn’t yet been addressed even though you were led to understand it would be?
LikeLike
It is a key question Margaret. The public need to have confidence in the satisfaction data and as you say, why are so many people satisfied when they have received no investigation and no uphold?
LikeLike
When I was in liaison with the PHSO I simply found them to be polite. You could feel them smiling in all correspondence, be it via telephone, email or letter, as they ignored obvious and vital evidence that, in my opinion, needed to be included to make the complaint process work.
I found the PHSO’s approach to gathering information in order to get to the truth very confusing. As a layperson I believed changing the scope, leaving out evidence, dragging out the complaint for over three years didn’t bode well for a fair and just investigation.
I was never asked to contribute to customer satisfaction surveys for the PHSO, and if I had of been asked I would have given them an honest account of my dealings with this organisation, and it would be that I believe the PHSO is simply a sham, not fit for purpose and a waste of taxpayers money. I’d advise people not to use the Ombudsman, it will seriously make what ever tragic circumstance led you to complain in the first place far worse.
I’d like to know why I’ve never been asked to take a satisfaction survey for the PHSO, it’s a mystery.
LikeLike