NHS gooroo blog

Posts Tagged ‘perverse’

Monitor is out to consultation on how it should judge NHS Foundation Trusts, and performance against the “18 weeks” targets is part of the mix. The deadline for responses is 4 April 2013, so this is a good opportunity to influence their proposed approach and fix its unintended consequences.

Monitor’s proposed approach

The place to go is page 78 of the consultation document. There are 28 targets and indicators, ranging from waiting times to C. Diff. to medication errors. Monitor propose that:

NHS foundation trusts failing to meet at least four of these requirements at any given time, or failing the same requirement for at least three quarters, will trigger a governance concern, potentially leading to investigation and enforcement action.

and each of the three 18-weeks measures is included as a separate indicator in the list:

• Maximum time of 18 weeks from point of referral to treatment in aggregate – admitted: threshold 90%
• Maximum time of 18 weeks from point of referral to treatment in aggregate – non-admitted: threshold 95%
• Maximum time of 18 weeks from point of referral to treatment in aggregate – patients on an
  incomplete pathway: threshold 92%

accompanied by the following note:

18 weeks referral to treatment: Performance is measured on an aggregate (rather than specialty) basis and NHS foundation trusts are required to meet the threshold on a monthly basis. Consequently, any failure in one month is considered to be a quarterly failure for the purposes of the Risk Assessment Framework. Failure in any month of a quarter following two quarters’ failure of the same
measure represents a third successive quarter failure and should be reported via the exception reporting process.

Will apply to consultant-led admitted, non-admitted and incomplete pathways provided. Failure against any threshold will constitute a governance failure. The measures apply to acute patients whether in an acute or community setting. Where an NHS foundation trust with existing acute facilities acquires a community hospital, performance will be assessed on a combined basis.

Monitor will take account of breaches of the referral to treatment target in prior quarters (i.e. under the Compliance Framework) when considering consecutive failures of the referral to treatment target under the Risk Assessment Framework

Response

As usual with consultations, it isn’t always easy to tell which of the consultation questions is the right one for any particular issue. For this response, I’ve gone for Question 10:

Chapter 4 Question 10: Do you agree with the proposed approach to downgrading the governance rating – and ultimately finding a foundation trust in breach – as a result of either unresolved concerns for significant periods or concerns across multiple categories?

No (with regard to the “18 weeks” access targets)

Please provide more details:

Appendix A includes the three “18 weeks” access targets, and each target is considered independently when assessing the number and duration of failures. Unfortunately this approach has unintended and undesirable consequences.

Consider the following scenario : a Foundation Trust has developed a backlog of over-18-week waiters on its waiting list. Because of this backlog, it starts to breach the incomplete pathways target.

The most desirable cure for the backlog is to treat those long-waiting patients as soon as possible. Unfortunately, the current approach deters the FT from doing so.

The reason is that if the FT admitted all those over-18-week waiters, it would be likely to incur a second breach (against the admitted patients target). FT Board members have a strong incentive to avoid this further breach (especially if they are also breaching any other measures) by using a well-established tactic: taking care to admit only one long-waiter in every ten admissions. This severely restricts their ability to do the right thing and treat the backlog, and the proposed approach is therefore a perverse incentive. Put another way, for every long-waiter they admit, they must find nine short-waiters to admit out of turn; this is unfair to patients, and all that queue-jumping pushes up the longest waiting times.

So what should be done?

Ideally the admitted and non-admitted targets would be deleted. But I anticipate that this suggestion might not be acceptable, because both of those targets are currently specified in the NHS Standard Contract 2013/14 as well as the regulations underpinning the NHS Constitution.

So I would like to propose an alternative solution: that the three 18-weeks targets should be lumped together into a single-failure bundle. So a failure against any (or all) of them would only be one failure in total. Then, if an FT is breaching on incomplete pathways, it would not incur any further breaches by treating its long-waiters, so the perverse incentive is removed. The intended incentive (to avoid long-waiters building up in the first place, monitored across all three measures) is nevertheless preserved.

In a very welcome last-minute change, the Commissioning Board has just amended the ‘final’ NHS Standard Contract 2013/14 and given top priority to clearing the long-wait backlogs on England’s NHS waiting lists.

There has been a dramatic turnaround in waiting times penalties during the drafting of this Contract. The ‘near final’ draft, published just before Christmas, perversely penalised hospitals for treating long-waiters, but not for allowing long-wait backlogs to build up in the first place (though it did introduce the new backstop penalties for having one-year waiters on the list). The supposedly-final version of the Contract, published on Monday, added new penalties for building up long-wait backlogs but gave them little weight. Today’s version of the Contract correctly slaps the highest penalties on the backlog, and reduces the legacy penalties for treating long-waiters.

As the final Contract stands now (Particulars p.58), any hospital specialty that allows more than 8 per cent of the waiting list (incomplete pathways) to exceed 18 weeks will be subject to a sliding scale of penalties up to 2.5 per cent of elective revenue. The older targets linger on, so that if they try to clear their backlog, and more than 10 per cent of the patients they select for admission have waited over 18 weeks, they face penalties up to 1.875 per cent of revenue. That is perverse, but it isn’t as bad as it sounds. Because the penalties are applied monthly, it is much cheaper to clear the backlog and pay the smaller penalty temporarily, than to let the backlog fester and pay the higher penalty indefinitely.

This fundamentally changes the incentives around waiting times, putting the emphasis firmly on avoiding backlogs rather than managing them. Nevertheless providers need to be aware that it is perfectly possible to achieve the ’92 per cent incomplete pathways’ target every month, and still consistently breach the ’90 per cent admitted patients’ target. When planning the list size that is consistent with sustaining all the 18-weeks targets (as sensible specialties do) it it best to plan against the most demanding one.

All this has felt like a very long journey. Waiting-list-based targets were first announced by Andrew Lansley as long ago as 17th November 2011, but disappointingly weren’t written into the subsequent NHS Standard Contract. Although the Mandate mentioned the waiting-list-based target as well as the treated-patient-based ones, it wasn’t clear about their relative priorities (and the waiting-list-based target was at a disadvantage because it wasn’t enacted in legislation until last week). But now it’s done, and the waiting-list-based targets have finally reached the top of the pile.

Why did it take so long? The main justification is that the incomplete pathways (waiting list) data is much more error-prone than the treated-patients data. When the last Labour Government introduced referral-to-treatment waiting times targets, it was a massive technical challenge to stitch together the waiting times of outpatients, diagnostic patients, and admitted patients, which in most hospitals are held on separate computer systems. It is easier to link the waiting times together towards the end of the patient pathway, once their activity has been coded from the early stages, than to link it together while they are still partway through. Nevertheless, data on incomplete pathways has been collected since August 2007, so I have to say I think the change could have been made earlier.

But we are there now, and it looks pretty good. The main penalties discourage over-18-week backlogs from building up, and in the coming months this should lead to further satisfying falls in long-waiters. We also have hefty zero-tolerance penalties where any patient is still waiting a year after referral, which should at long last bring those extreme long-waits to an end. With the focus returned to the waiting list where it belongs, providers are now encouraged to focus on the fundamentals: keeping the list size down and scheduling patients in the right order. That’s better for patients, better for the service, and much less confusing for the public.

The final 2013/14 NHS Standard Contract has now been published, with a small but welcome change in the penalties for breaching the 18 week waiting times targets. In the previous “near-final” draft there were penalties for treating long-waiters, but none for allowing long-waiters to build up in the first place.

In the final version (Particulars p.58), any specialties with large 18-week backlogs face new penalties of up to 0.625 per cent of elective revenue every month. But the penalties for admitting long-waiters also remain in force: if a specialty admits its long-waiters instead of keeping them waiting, it faces penalties of up to 2.5 per cent of elective revenue every month (although those higher penalties would only apply while the backlog is in the process of being cleared).

It is welcome that the Commissioning Board has introduced penalties for having 18-week backlogs. This tackles the root of the problem, and rightly draws attention to the waiting list itself instead of those patients lucky enough to be selected for treatment. It may also strengthen the hand of providers who wish to treat extra patients in order to control the size of their waiting lists.

However at 0.625 per cent the penalties are rather small, and the regime is now very complex with contradictory penalties applying across the three 18-week targets. This makes it difficult for people who are not immersed in the subject to understand and interpret the numbers (a problem that has extended right up to the Prime Minister). The continued penalties for treating long-waiters are perverse, and it would be better to drop them and simply monitor completed patient pathways as a means of catching data errors and ‘gaming’.

So it’s a small and rather slow step, but at least it’s in the right direction.

How should the NHS respond to the new target regime? Fundamentally, waiting times are a function of the size of the waiting list and the order in which patients are scheduled. It is no longer possible for specialties to avoid penalties simply by admitting 9 short-waiters for every 1 long-waiter, and instead they must address the fundamentals: by knowing how small their waiting list needs to be to sustain 18 weeks, and keeping it below that size; and by scheduling patients according to the well-accepted principles that urgent patients should be treated quickly and other patients should be treated broadly on a first-come-first-served basis.

If the new penalty regime achieves a widespread return to those fundamentals, then it will have succeeded.

In the last blog post we looked at the unexpectedly widespread use of clock pauses in helping Trusts scrape under the adjusted admitted waiting times target. In this post we’ll drill down into the local data, to see where clock pauses are used the most.

This interactive map shows, by Trust and specialty, by how many weeks clock pauses are reducing the reported 90th centile adjusted admitted waiting times. You can zoom in on the map to get more detail, and when you get down to pin level you can click on the pin to get a multi-page balloon showing the numbers (scroll through the pages using the arrow at the bottom right corner of the balloon).

Here is the same chart on a PCT basis. And here are tables with the raw data: Effect of clock pauses by Trust

The titles in the data balloons are also clickable, and they take you to a more detailed analysis of waiting times in the service, including a snapshot of the latest position and time trends showing how performance has varied in the last three years.

For instance, this time trend shows how clock pauses started being used more regularly as time went by (in one Trust’s Orthopaedics service). The heavy red line shows the unadjusted waiting times creeping up, and the thin red line with squares on it shows the adjusted waiting times clinging to the target:

We have added clock pauses to our regular monitoring, so you’ll be able to check the local position every month from now on in this blog.

The Department of Health’s draft Mandate to the new NHS Commissioning Board was published last week, and it’s bad news for anybody hoping for a bit of common sense on the 18-week waiting times targets. All three targets are being retained: that’s one target telling hospitals to treat their long-waiting patients, and two targets punishing them if they do.

The good news is that this is a consultation draft, so you have until 26th September to tell the Department of Health why they should reconsider. As usual, with government consultations, you have to find a way to fit your comments into a pre-defined set of strangely-tangential questions, and in this case the one to use is question 3: “Are the objectives right?”.

I am sure that staff in many Trusts will be able to provide the Department with plenty of examples from their own experience of how the offending targets have distorted patient care, confused and misled stakeholders on the waiting times position, and resulted in unfair criticism and financial penalties, when all they are trying to do is the right thing for patients.

Here is the response I am submitting:

Question 3. Are the objectives right? Could they be simplified and/or reduced in number; are there objectives missing? Do they reflect the over-arching goals of NHS commissioning?

Objective 10 specifically mentions three service performance standards for referral-to-treatment (RTT) waiting times. An NHS Trust with few long-waiters will achieve all three performance standards, and one with many long-waiters will not. However that does not mean they are all good standards to use, and the use of all three is already having unintended consequences for patients in those parts of the NHS where a backlog of long-waiters has built up.

Specifically, two of the performance standards (that 90% of admitted and 95% of non-admitted patients must start treatment within a maximum of 18 weeks from referral) are often detrimental to waiting times performance, undermine the ability of the NHS to deliver the NHS Constitution right to treatment within 18 weeks, and are unfair both to patients and to NHS Trusts; these two standards should be omitted. The third RTT standard (that 92% of incomplete pathways should have been waiting no more than 18 weeks from referral) should be retained.

The following example illustrates the point.

Trust A has developed a backlog of patients on its waiting list who have already waited over 18 weeks. The Trust does not want to have a backlog, and notes that the NHS Constitution right to treatment within 18 weeks, the incomplete pathways performance standard, the accepted principle that patients with similar clinical priority should broadly be treated on a first-come-first-served basis, and the wishes of clinicians and managers alike, all point towards a clear and simple solution: treat the over-18-week waiters and thereby clear the backlog.

However Trust A is restricted from doing so by the admitted patients performance standard, which stipulates that 90% of admitted patients must be selected from those who have waited less than 18 weeks. (The performance standard for non-admitted patients has exactly the same effect, though in practice it is less likely to be the stumbling block.)

The admitted patients performance standard has a number of effects:

1) In order to clear 100 long-waiting patients who have already breached 18 weeks, the Trust must at the same time admit 900 short-waiting patients whether their clinical priority justifies it or not (and in most cases it will not). This queue-jumping is unfair to the long-waiting patients.

2) This queue-jumping also pushes up maximum waiting times (as queue-jumping does in any queue) thereby making the long-wait backlog worse than it would have been without the queue-jumping. The number of over-18-week waiters will therefore be much higher than it would have been, if the Trust had been allowed to treat non-urgent patients in date order. This undermines the NHS Constitution right to treatment within 18 weeks.

3) Put another way, the Trust is only able to clear the backlog slowly, because it is only allowed to devote 10% of its activity to the long-waiting backlog. If this restriction were lifted, it could devote all its non-urgent capacity to the backlog (typically between 50% and 95% of activity depending on the number of urgent patients in the casemix) and clear it much more quickly.

4) The Trust Board’s monthly Performance Report monitors all three performance standards, but a majority of Board members have a limited understanding of how the standards act in opposition to each other. This leads to poorer monitoring and decision-making than if just one performance standard, whose effect is intuitive, were monitored.

5) Statistics are collected and published nationally based on all three performance standards, which leads to misunderstandings about the NHS’s waiting times performance by the general public, journalists, and politicians. Such misunderstandings have in the past reached the highest level: exchanges between the Prime Minister and Leader of the Opposition at Prime Minister’s Questions have on occasion assumed that an increase in the number of long-waiters being treated is a bad thing, when in fact it resulted in a reduction in the number of long-waiters still waiting which is a good thing.

In contrast to the admitted and non-admitted performance standards, the newer standard (that 92% of patients on incomplete pathways should have been waiting no more than 18 weeks from referral) does not similarly frustrate good waiting list management, and should therefore be retained.

It may be that the admitted and non-admitted performance standards were included in Objective 10 because they are referred to in directions 2 and 6 of The Primary Care Trusts and Strategic Health Authorities (Waiting Times) Directions 2010, which were intended to support the 18 week rights in the NHS Constitution. For the reasons given above they are poor instruments for delivering that intention, and the directions should therefore be amended to omit reference to the admitted and non-admitted performance standards, and refer instead to the 92% incomplete pathways standard.

Polly Toynbee’s piece on waiting list fiddles attracted a lot of attention, and quite right too. Within hours of being published, it was even being mentioned in the House of Commons at Health Questions.

Toynbee reported that:

The national target says 90% of patients must be treated within 18 weeks of first referral

and this has unpleasant consequences. A waiting list clerk, who resigned on principle, said:

She was told to cancel operations for anyone who was already waiting over 18 weeks, and instead to fill that theatre time with people closest to breaching the 18-week limit.

and

She was told not to book anyone already in breach until April and the start of the next financial year, or to book only one for every nine still under the target. Instead she was told to fill theatre slots with as many short, minor operations as possible.

This happens. It shouldn’t, but it does. The immediate and natural reaction is to blame the managers who gave the orders, but they are also just pawns in the game.

Imagine you are responsible for delivering the target for surgery in an acute general hospital. It’s now late February, so all the operating lists are nearly full for March, and new bookings are being made into April.

Let’s say you have booked 1,000 patients in March; 895 will have waited 18 weeks or less on the day of their operation, and 105 will have waited more than 18 weeks. The target says that 90 per cent must have waited less than 18 weeks, and you are heading for failure with only 89.5 per cent. What do you do?

You don’t have many options. If you breach the target, your Trust may receive a heavy fine from commissioners under the standard NHS Contract, and be subject to “performance management” by the SHA or Monitor; and you personally may be summoned to “explain yourself” to the Director of Operations or Chief Executive. Not an attractive option.

Or you could book in some extra under-18-week patients to bring performance up to 90 per cent. But that means finding operating time for 50 extra patients, and your lists are already nearly full. To make matters worse, you have used this tactic before and are running out of minor operations to pad out the target. Finances are tight, and you can’t afford to pay extra for lots of Saturday lists. So this option is not attractive either (and by using this option in the past, you have made today’s problem worse because your waiting list is now skewed towards heavier cases).

Which makes you think: you’re only 5 patients adrift of the target. If you just put 5 long-waiting patients off until April, and recycle their operating time for shorter-waiting patients, then you’re done. If you can choose five patients who are just about to breach 18 weeks then you have headed off your next problem too…

You can see how easy it is. Yes, it’s wrong, but in the circumstances what else can you do?

Following the story up at national level, Toynbee said: “Professor John Appleby of the King’s Fund health thinktank says he hears of waiting-list cheating from many hospitals and will suggest the National Audit Office investigates” and she asks whistleblowers to get in touch with her. Both are excellent ideas.

But we need to take care not to turn this into a punitive witch-hunt against cheating hospitals, because the targets and associated performance management are the root causes. So instead of prolonging the damaging treatment-based waiting time targets for another year, let’s abandon them now and move straight to the new waiting-list-based target instead. That, at a stroke, would eliminate the main cause of the problem.