NHS gooroo blog

Archive for January, 2011

Most Trusts are achieving the 18 week operating standard. But some aren’t, and others are slipping towards failure. What should they do about it? Throw scarce money at it by creating extra capacity? Or can the problem be fixed more cheaply by managing better? And how can they tell which is the right approach?

We are currently writing to English Trusts, indicating their Trust’s likely best approach in all main surgical specialties. This post puts those individual reports into context, showing the national picture and explaining in more detail how the reports were constructed.

Here is a chart showing General Surgery at English hospital Trusts. (The format was introduced in a previous post.)

Three Trusts have been picked out in different colours. Here is the distribution of referral-to-treatment (RTT) waiting times for the red Trust:

The waiting time report being sent to this Trust says (for General Surgery):

• Achieving 90% within 18 weeks? No – only achieving 70%
• Clinical priorities under pressure? Indication of some pressure to delay urgent patients (proportion admitted within 4 weeks RTT is in lowest quartile)
• Scope to reduce waiting times by better scheduling? Possible scope for significant improvements at modest cost by improving scheduling

Looking at the charts, we can see where these statements come from.

• On the bubble chart, the red bubble lies well to the right of the “90% within 18 weeks” line, and is out at 30% breaches (i.e. 70% within 18 weeks), showing that the 18 week operating standard is not being met.
• Looking at the vertical position of the red bubble, we can see the proportion of patients treated within 4 weeks RTT: this Trust lies between the bottom line (lowest decile) and second-bottom line (lowest quartile), so the Trust’s report states that there is an indication that clinical priorities might be under pressure, because the proportion treated quickly is in the lowest quartile.
• Looking at the red column chart, we can see that a significant proportion of patients are admitted with intermediate waiting times from 4 to 15 weeks. Because these cohorts form a majority of the non-urgent patients, the Trust’s report states that waiting times might be reduced by improving scheduling. This statement is only suggestive, because there may be good reason (such as subspecialisation between consultants preventing waiting time pressures from being shared across the specialty) why better scheduling might have limited impact.

For the Trust picked out in amber, their waiting time report says:

• Achieving 90% within 18 weeks? Only just – 92%
• Clinical priorities under pressure? No indication of pressure to delay urgent patients
• Scope to reduce waiting times by better scheduling? Likely scope for major improvements at modest cost by improving scheduling

because

• The amber bubble is only just to the left of the 90% line.
• The bubble is above the lowest quartile (second-from-bottom line), so there is no indication (based on the proportion treated within 4 weeks RTT) that clinical priorities are being squeezed.
• Lots of patients are being treated with intermediate waiting times – neither as urgents nor as 18-week pressures.

And here is the green Trust:

This Trust’s report says:

• Achieving 90% within 18 weeks? Yes – 97%
• Clinical priorities under pressure? No indication of pressure to delay urgent patients
• Scope to reduce waiting times by better scheduling? Possible scope for significant improvements at modest cost by improving scheduling

It so happens that all three examples have indicated scope to reduce waiting times by better scheduling, but this is not always the case. In General Surgery we indicate “likely” scope for 45% of Trusts, “possible” scope for 25% of Trusts, and “limited scope” for the remaining 30% of Trusts.

Finally, here are the bubble charts for the other main surgical specialties.

First law of forecasting

Forecasts are always wrong.

Second law of forecasting

Detailed forecasts are worse than aggregate forecasts.

Third law of forecasting

The further into the future, the less reliable the forecast will be.

Factory Physics, p.441

So if all forecasts are wrong, why bother? Well, the “first law” is a bit mischievous; instead of “wrong” perhaps “inaccurate” would be closer to the truth. As a Professor of Statistics once said:

All models are wrong but some are useful.

George Box

We cannot avoid forecasting. Even if we refuse to make explicit forecasts, and just carry on as usual, then we are effectively forecasting that the future will be like the past. So we make forecasts because we expect the future to be different in some way, or because we expect the analysis to tell us something useful that we don’t already know… or perhaps because someone told us to.

All forecasting starts by estimating future demand, and in healthcare there are two main ways of doing this. We could look at population, morbidity, medical advance, and anything else we can think of, and try to work out from first principles how much demand there should be for healthcare. Try it if you like, but you’ll be massively and embarrassingly wrong. The better alternative is to start by looking at actual demand in the recent past, and estimating how it might be affected by future trends.

And how do we measure demand? In theory we want to get as close to the source of demand as we can: which from a GP Commissioner’s point of view means evaluating all contacts between primary care practitioners and patients; and from an acute hospital’s point of view means evaluating GP and consultant referrals and A&E arrivals. Which is all very well, but in practice does not give us a complete enough picture; we don’t know what is wrong with patients when they first arrive, and so we don’t know what activity will be needed to care for them. So in healthcare, we end up using activity as a proxy for demand.

Starting with observed activity as our baseline, we then apply some kind of trend growth rate. This trend might indeed be based on demographics and medical advance (but these usually underestimate growth by a large margin), or worked backwards from financial affordability (which at best shows the scale of the challenge facing us, or at worst is merely wishful thinking), or simply estimated by looking at what happened in recent years (which is pragmatic and usually best).

Whichever method we pick, it is still going to be either inaccurate or a fluke. No trend continues forever, and these errors in future demand trends are a big source of error in any healthcare forecasting model. The more detailed we make our plan (HRGs, monthly profiles…), the more volatile the numbers; the further into the future we go (25-year PFI capacity plans…) the worse our trend assumptions. The second and third laws of forecasting are right about all that.

Given the inaccuracies around demand, there is little point in being over-sophisticated about the rest of the forecast. But there are a few other things that make a big enough difference to matter:

• If we’re using part-year historical data in a highly-seasonal area such as medicine or trauma, then we need to smooth it for the seasonal effects to make the baseline representative. (Though it’s usually easier just to use a full year’s data.)
• If we’ve been doing a lot of non-recurring activity (or failing to keep up with demand) in the past, then we need to adjust our baseline demand accordingly.
• What if there are specific things we know we are going to change, such as diverting COPD patients to a primary care led service, ceasing a low-effectiveness treatment, or stopping activity that does not address demand? The best way to handle these is to change the baseline activity as if the change were already in place.

Other than correcting for those kinds of things, the emphasis of our forecasting should not be on trying to improve accuracy any further: we have done enough.

Instead we should focus on making our forecasting useful. What capacity will providers need? What will waiting times be? How much will it cost? Where can we disinvest? How should we present the results so that we can understand them and take the right action?

There is another benefit to keeping forecasting simple and pragmatic: it makes it easier to relate our high-level longer-term forecasts to our more-detailed and shorter-term operating plans. By adopting common assumptions, when reality doesn’t turn out quite the same as our forecast and our operating plans are adapting, at least we can relate our local knowledge more easily to the big picture.

Hospitals accepting unselected medical emergencies must have on-site surgery.

Acute health care services: Report of a Working Party.
Academy of Medical Royal Colleges, September 2007, p.22

There’s a lot of guidance like this, from the Royal Colleges, subspecialist societies, NCEPOD, and the Department of Health, all describing in helpful detail the critical links that exist between different acute services.

But each document describes only a few strands in a complex web of interdependencies. Senior clinicians and managers, however, need a system-wide view, but it is difficult to piece together the whole picture from this mass of detail.

The lack of a big picture can waste a lot of time. When acute reconfigurations are being considered, managers and clinicians may get together in a large group to draw up the reconfiguration options. Much later, after a lot of work, some options have to be struck out when a fatal flaw is discovered (such as not being able to separate paediatrics from obstetrics). At worst the lack of a big picture can be dangerous, when piecemeal changes are made locally, to individual services, without realising that they could destabilise the whole hospital.

So we need an overview of these important clinical linkages. Looking only at those 24-hour services that must be provided on the same hospital site, we think the links look something like this. A solid line means that one service must support the service it points to; a dotted line means that it is possible to run the service without that support but procedures must be in place to ensure safety.

There are caveats of course. It isn’t possible to capture all the nuances of this complex guidance in one diagram: for instance, the distinction between a selected and unselected acute medical take is not fully captured. Also there are cases where older guidance states a requirement that is not mentioned in more recent, overlapping guidance; this leaves it unclear whether the requirement has been softened. In the full version of this work, therefore, the diagram is accompanied by the relevant passages from the guidance (referenced by the numbers beside the arrows).

We think this is the first time that acute clinical linkages have been comprehensively published in this way. At a time when acute hospitals and commissioners are under pressure from the EWTD and the financial squeeze, many are considering whether they could transfer services to an adjacent hospital or stop providing them altogether. This map of acute interdependencies should help to show where this can, and cannot, be done safely.

To take just one example, could you save money by downgrading physiotherapy to daytime only? A novice manager might think so. But the answer is clearly no, because that would put at risk the intensive care unit, acute surgery and medicine, and the A&E department. Not a career-enhancing move.