Phase 1 of the NPE demonstrated positive participation, achieving an 81% return rate from NHS organisations which in turn provided substantial basis for analysis work undertaken to establish the baseline indicator.
The success of phase 1 and positive engagement encouraged a 10% increase in new NHS organisations participating in the comparison dataset exercise (2019 – 2020) for phase 3 of the project.
NHSCFA obtained statistical support from NHS Counter Fraud Scotland. A peer review was undertaken of the methodology and analysis used to determine the most vulnerable instances of non-PO spending and the identified FVE decrease between the first and second exercises for the 79 organisations to submit data for both.
The methodology required the collection of two datasets of all spend and spend that did not link to a PO. The NHS-eClass system was used to classify spend.
The baseline dataset (2018 – 2019) contained submissions from 94 organisations. The comparable dataset (2019 – 2020) contained submissions from 211 organisations. Due to an identified anomaly in the comparable dataset, one NHS organisation’s data was removed from analysis. There were 14 of the original 94 organisations that did not take part in this new exercise to provide comparable data. Therefore, conclusions (with regard to the fraud prevention campaign’s impact on FVE) are drawn from data provided by the remaining 79 organisations that participated in both the baseline and comparable data collections.
Organisation-specific feedback will however be provided to all 211 organisations, including those that did not participate in both data collection exercises (baseline and comparable). To enable this, we assessed the overarching total spend vs nonPO spend figures for 2019 – 2020 to showcase each organisation’s performance in relation to the complete dataset. It should be noted that these calculations were deemed too vague to be utilised as part of our analysis for identifying impact on FVE. However, this overarching data can be used as a benchmark by which each of the 211 organisations can see how they have performed in relation to the overall average.
The below table shows total spend, non-PO spend and non-PO average for the 210 organisations across all 23 spend categories in all four quarters of 2019 - 2020.
| Organisational breakdown (2019 – 2020) | Total spend | Total non-PO total spending | Non-PO average |
|---|---|---|---|
| 79 organisations | £9,372,012,323.65 | £4,342,284,777.24 | 46.33% |
| 131 organisations | £40,326,665,928.01 | £10,718,786,927.69 | 26.58% | 210 organisations | £49,698,678,251.66 | £15,061,071,704.93 | 30.30% |
The 79 organisations to submit data had 7,268 ‘opportunities’ (23 categories in each of four quarters) to conduct non-PO spend. Due to the high volume of raw data, a decision was taken to focus on instances where organisations exceeded non-PO spend by 90% or more on at least one occasion and in (eClass) categories of spend with an overall average of 30% or more across each quarter. By using this methodology, our analysis has identified pertinent trends, the most vulnerable areas of spend, and those organisations most responsible for non-PO spend.
This methodology was originally used to undertake analysis on the original baseline dataset from 2018 – 2019 and also applied to analysis undertaken on data collected from the 2019 – 2020 financial year. By using the same methodology across both datasets, comparative conclusions can be drawn on the most vulnerable areas of spend.
As only 79 organisations took part in both data collection exercises, it was therefore necessary to reconfigure analysis on the baseline data, using the same methodology, to reflect activity for those organisations.
This methodology identified seven main vulnerable (eClass) categories of spend across the two data collections:
- D: Pharmaceuticals Blood Products & Medical Gases
- L: Fuel, Light, Power, Water
- M: Hotel Services Equipment Materials & Services
- P: Building & Engineering Products & Services
- R: Purchased Healthcare
- X: Transportation
- Z: Staff & Patient Consulting Services & Expenses
To avoid under or over emphasising figures, the statistician recommended that from this point on, to also count spend where certain occasions did not meet the 30% threshold per quarter (as set out in paragraph. 58). This would ensure that spend that fell under the 30% threshold would be accounted for and therefore build an accurate picture of FVE.
Two of the seven categories (Category L: Fuel, Light, Power, Water and Category R: Purchased Healthcare) were excluded from FVE calculation. Engagement with the sector has confirmed that spend in these two areas are never fixed costs in comparison to other types of spend and are usually processed without a PO number. Therefore, they could never have realistically been impacted by any prevention measures the NHSCFA or local NHS organisation put in place.
Analysis has been undertaken to assess the differences in FVE between the baseline and comparable datasets to assess the impact of fraud prevention activity undertaken during the NPE.
Statistical advice undertaken by the NHSCFA has concluded that the calculation of confidence intervals based on the FVE estimate would be technically complex to extrapolate findings of the sample of 79 organisations to the remaining 131 organisations that did not participate in both data collection exercises (baseline and comparable). There is not an obvious standard methodology that could be applied to an extrapolation calculation. However, the statistician has provided assurance of statistical significance on the reporting sample and on the analysis and findings on FVE calculations.