This poster highlights the challenges encountered during large-scale implementation of an artificial intelligence (AI) assist device for chest radiograph (CXR) interpretation across a national radiology network, how these challenges were overcome with careful planning, and how pre-determined success metrics were useful with regards to evaluating feedback for the device.

Background: AI has enormous potential to enhance performance in clinical radiology, and ultimately improve patient outcomes in our healthcare system.[1] Many AI models have been validated for their diagnostic performance,[2]however successful real-world implementation of an AI assist device into clinical practice is a complex, multifaceted task requiring further evaluation. First and foremost, an AI device must positively impact radiologists’ performance and reporting experience, without hindering their overall efficiency. Seamless integration into existing radiology information system (RIS) and picture archiving and communication system (PACS) systems is...

Pilot Study: The 11 radiologists in the pilot study reported a total of 2972 CXR cases from 2665 patients, with accompanying feedback. The male:female distribution across the cases was 52.7% : 47.3%. The per-case feedback data acquired via the modified device user interface showed that in 3.1% of reports (92 studies), significant interpretive changes were made that were attributable to the device. In almost half of these cases (1.4% of total cases) the AI device’s interpretation led to changes in patient management. In 1.0% of...

For AI assist devices to positively impact healthcare, as intended by their developers, successful implementation into clinical practice is of utmost importance. Integration into existing RIS and PACS systems, across multiple sites, with varying clinical settings and a diverse mix of users, is inherently complex. Through careful planning the use of pre-determined success metrics, we have demonstrated how successful device implementation can be achieved. Results from both the pilot study and the post-implementation survey indicated that the AI device was very well integrated into the...

Thrall JH, Li X, Li Q, Cruz C, Do S, Dreyer K, Brink J. Artificial intelligence and machine learning in radiology: opportunities, challenges, pitfalls, and criteria for success. Journal of the American College of Radiology. 2018 Mar 1;15(3):504-8. Aggarwal R, Sounderajah V, Martin G, Ting DS, Karthikesalingam A, King D, Ashrafian H, Darzi A. Diagnostic accuracy of deep learning in medical imaging: A systematic review and meta-analysis. NPJ digital medicine. 2021 Apr 7;4(1):1-23. Huisman M, Ranschaert E, Parker W, Mastrodicasa D, Koci M, Pinto de...