The Royal College of Radiologists (RCR) works closely with individual radiology departments across the four countries within the UK; nominated individuals/Fellows (audit leads) within the departments are responsible for co-ordinating both local audits and national RCR audit projects.
All audit leads were invited to submit data in both non-surgical and surgical cohorts to the RCR on behalf of their departments.
For each cohort,
a retrospective search of the radiological department database was undertaken to identify 25 consecutive non-traumatic adult (>16 years) emergency patients who had undergone abdominopelvic CT from 1st January 2013 onwards.
The patients all had out of hours (6pm-8am weekdays or anytime at the weekend) emergency abdominal or abdominopelvic CT,
but the non-surgical group had no subsequent laparotomy whereas the surgical group did.
Auditors,
who were were blinded to the original report content/reporter, were instructed to select the provisional (+/- addendum) report diagnosis,
recording their own auditor diagnosis if non-concordant and then also record the laparotomy diagnosis.
This process allowed recording of major/minor discrepancy between reports by the auditor,
also the type of discrepancy and,
using patient/radiology records,
to assess any harm that may have come to the patient.
Correlation with surgical findings was also undertaken.
Each auditor was a substantive consultant working onsite in the auditing institution and had experience in reporting abdominal CT.
A major discrepancy was defined as a change or potential change in diagnosis or treatment as a result of either addendum report or CT auditor review.
A minor discrepancy occurred where there were minor issues in provisional/addendum reports unlikely to result in harm or change in management.
Audit Standards
The selected audit standards,
together with compliance,
are included in Table 1.
A search of all available published literature (from 1950 onwards) was undertaken using the MEDLINE and NHS (National Health Service) Evidence (including the Cochrane Library of Systematic Reviews and the National Library of Guidelines) to establish supporting literature and confirm/derive figures for the audit standards.
There were three main groups providing provisional (initial) CT reports:
- Registrar (trainee radiologist)
- Offsiter (radiologist working remotely for an outsourcing agency at consultant level or equivalent)
- Consultant radiologist onsite (may or may not have GI radiology expertise)
Addendum (supplementary) reports to provisional reports are provided by hospital-based onsite consultant radiologists with varying degrees of GI radiology expertise.
Results
A total of 109/188 eligible departments responded to the audit (58%).
Departmental demographics are included in Table 2 and case demographics in Table 3.
4931 patients were included in the audit of which 2568 were in the non-surgical group and 2363 in the surgical group.
48% were male and 52% were female.
Discrepancy Rates
Non-Surgical Group
1947 patients had a provisional CT report with no evidence of addendum and of these there was concordance with the auditor in 1782 patients.
621 patients had evidence of an addendum report with provisional,
addendum and auditor reports concordant in 472 patients.
Varying levels of discordance were noted in the remaining patients with the most prominent category being auditor concordance with addendum and not with provisional (75 patients).
Surgical Group
1728 patients had a provisional CT report with no evidence of an addendum and of these the provisional report was concordant with the auditor in 1557 patients.
In 1423/1557 there was also agreement with laparotomy.
635 patients had evidence of an addendum report with provisional,
addendum and auditor reports concordant in 510 of these patients.
Varying levels of discordance were noted in the remaining patients with again the most prominent category being auditor concordance with addendum and not with provisional (72 patients).
In the 510 patients with concordance of all 3 reports there was also agreement with laparotomy findings in 471 (39 disagreed).
Major Discrepancies and Patient Harm
Non-Surgical Group
In 47 patients there was evidence on notes/imaging review of subsequent additional procedures that may have been unnecessary following a major discrepancy.
These were predominantly additional imaging procedures,
but also included CT/ultrasound guided drainage (3 patients),
laparoscopy (3 patients) and endoscopy (3 patients).
15/72 patients with provisional report major discrepancy were considered by the auditor to have come to harm as a result of the report: Delay in diagnosis (7 patients),
delay in treatment (7 patients),
unnecessary investigations (2 patients) and unspecified (1 patient).
Surgical Group
36/132 patients with provisional report major discrepancy were considered by the auditor to have come to harm as a result of the report and were detailed as follows: Delay in diagnosis (3 patients),
delay in surgery (24 patients),
unnecessary investigations (1 patient) and unnecessary surgery (8 patients).
Factors Affecting Major Discrepancy Rate
Table 4 shows risks of major discrepancy for onsite consultants,
radiology registrars and offsite reporters separately in the surgical and non-surgical groups.
Overall risks of major discrepancy were 5.6% in the surgical group and 2.8% in the non-surgical group.
In each group major discrepancy risks were highest in offsite reporters and lowest in onsite consultants,
although these between group differences only achieved statistical significance in the surgical group (p=0.0003).
In the combined analysis,
major discrepancy risks were 44% higher (95% CI 5% lower to 118% higher) in registrars than onsite consultants and 181% higher (95% CI 75% to 351% higher) in offsite reports than registrars (p=0.0001,
joint test of differences).
Among consultants,
discrepancy risks were lower in those with a GI interest or a GI sub-specialty than in those without such specialisation.
Combining the two specialist groups,
risk of a major discrepancy was 28% lower (95% CI 57% lower to 21% higher) and risk of discrepancy was 32% lower (95% CI 5% to 51%),
with this latter difference achieving statistical significance (p=0.022).
Factors Affecting Cases of Major Discrepancy where Patients Came to Harm
Restricting to major discrepancies where the patient came to harm,
numbers were reduced but the pattern of results was similar; for the pooled analysis the joint test of differences among the three groups was borderline statistically significant (p=0.061) with risks statistically significantly higher for the offsite group compared to the onsite consultants (p=0.018).
Added Value of a Consultant Addendum Report
In the non-surgical group,
there is net benefit from switching to an addendum report in terms of major discrepancies (19 resolved,
3 introduced) and in terms of all discrepancies (75 resolved,
26 introduced).
Using conditional logistic regression both differences are statistically significant (p=0.006 major discrepancy,
p<0.0001 all discrepancies).
In the surgical group,
there is again a strong net benefit in switching to an addendum,
both in terms of major discrepancies (45 resolved,
2 introduced) and all discrepancies (72 resolved,
13 introduced).
Using conditional logistic regression both differences are statistically significant (p<0001).
Availability of Provisional and Addendum Reports Pre-Operatively
A written or validated RIS (Radiology Information System) provisional report was available pre-operatively in 98.3% of patients (Table 1).
A written or validated addendum report was only available pre-operatively in 64.3% of patients.
In 45 patients with a major discrepancy at provisional report level the discrepancy was corrected at addendum.
In 14/45 of these cases the addendum was not available pre-operatively; hence there were 14 cases of potentially avoidable major discrepancy (only 1/14 patients came to harm).
Overall Compliance with Audit Standards
From Table 1,
it can be seen that the audit standard for provisional report major discrepancy was achieved for registrars (target <10%,
achieved 4.6%),
for onsite consultants (target <5%,
achieved 3.1%) and consultant addendum (target <5%,
achieved 2.9%).
Offsite reporters failed to meet the standard target (<5%,
achieved 8.7% overall and 12.7% in surgical patients).
The standard for patients coming to harm was not met in the surgical group (target <1%,
achieved 1.5%) and was narrowly missed in the overall (target <1%,
achieved 1%).