This study provides insight into radiologists’ feedback and perceptions following the implementation of an artificial intelligence (AI) assist device that comprehensively detects radiologic findings on chest radiograph (CXR) into a national-scale radiology network. We present data from an initial pilot investigation, as well as feedback from radiologists two months post wider network implementation across a diverse range of clinical practices.

Background AI has the potential to significantly advance clinical radiology and improve patient outcomes through radiologists’ diagnostic accuracy, and thereby reductions in the rate of missed findings of importance in imaging studies [1, 2]. Most studies published to date evaluating AI devices, however, are conducted in non-clinical, experimental settings, without means of measuring impact on clinical outcomes and overall benefits to healthcare systems. Furthermore, positive impacts of AI device usage established in experimental settings may not necessarily translate to improved performance in the real world....

Results of Pilot Study: In total, 2,972 CXR cases, from 2,665 unique patients, were reported during the pilot study, with a median patient age of 67 (IQR 50-77) and a 52.7% male to 47.3% female distribution. Based on per-case feedback, radiologist agreement with AI device predictions was high with complete agreeance with the AI device findings in 2,572 cases (86.5%). There were 306 cases (10.2%) where the radiologists rejected one finding by the AI device, and 84 cases (2.8%) where radiologists rejected two or more...

Radiologists’ feedback, reflected through results from the pilot study, as well as feedback two months post wider implementation throughout the radiology network, indicated high satisfaction with regards to device accuracy, impact on CXR reporting, user training and integration into existing workflows. The pilot study data showed that a significant number of CXR reports contained changes attributable to use of the AI device. Notably, this proportion was higher for radiologists with less than 5 years consultant experience, and there was also a correlation between number of...

S. Karunasena: Employee: I-MED Radiology M. Milne: Employee: Annalise.ai M. G. Vasimalla: Employee: Annalise.ai L. A. Danaher: Employee: I-MED Radiology M. Wilson: Employee: Annalise.ai Q. Buchlak: Employee: Annalise.ai C. M. Jones: Employee: Annalise.ai Employee: I-MED Radiology

Thrall JH, Li X, Li Q, Cruz C, Do S, Dreyer K, Brink J. Artificial intelligence and machine learning in radiology: opportunities, challenges, pitfalls, and criteria for success. Journal of the American College of Radiology. 2018 Mar 1;15(3):504-8. Yi PH, Kim TK, Yu AC, Bennett B, Eng J, Lin CT. Can AI outperform a junior resident? Comparison of deep neural network to first-year radiology residents for identification of pneumothorax. Emergency Radiology. 2020 Aug;27(4):367-75. Strohm L, Hehakaya C, Ranschaert ER, Boon WP, Moors EH. Implementation of...