The proposed DLAD (Carebot AI CXR v2.00) has achieved significantly higher Se (0.910 (0.854-0.966)) than that of all assessed radiologists (RAD 1 0.290 (0.201-0.379), p < 0.001, RAD 2 0.450 (0.352-0.548), p < 0.001, RAD 3 0.670 (0.578-0.762), p < 0.001, RAD 4 0.810 (0.733-0.887), p = 0.025, RAD 5 0.700 (0.610-0.790), p < 0.001), and would therefore help to identify patients with pulmonary lesions which the radiologists evaluated as without the suspicious finding.
However, the DLAD achieved a lower Sp (0.775 (0.717-0.833)) than all assessed radiologists (RAD 1 1.000 (0.984-1.000), p < 0.001, RAD 2 0.970 (0.946-1.000), p < 0.001, RAD 3 0.980 (0.961-1.000), p < 0.001, RAD 4 0.975 (0.953-0.997), p < 0.001, RAD 5 0.995 (0.985-1.000), p < 0.001), and the difference was statistically significant. The higher false positive rate of the DLAD was expected since the threshold was initially set to classify even suspect findings as abnormal. In particular, rib summation and more prominent pulmonary vascular markings were among the problematic regions that indicated multiple false positives.
The DLAD PLR (4.044 (3.104-5.269)) was significantly lower (i.e., worse) than that of radiologists (RAD 1 N/A (no CXR was evaluated as FP), RAD 2 15.000 (6.624-33.966), p = 0.002, RAD 3 33.500 (12.575-89.245), p < 0.001, RAD 4 32.400 (13.564-77.389), p < 0.001, RAD 5 140.000 (19.734-993.174), p < 0.001), and the NLR (0.116 (0.062-0.218)) was lower (i.e., better) than that of radiologists (RAD 1 0.710 (0.626-0.804), p < 0.001, RAD 2 0.567 (0.474-0.678), p < 0.001, RAD 3 0.337 (0.254-0.445), p < 0.001, RAD 4 0.194 (0.130-0.292), p = 0.132, RAD 5 0.301 (0.223-0.406), p = 0.003), and the difference was statistically significant, with the exception of RAD 4, where no statistical difference was found for this parameter (p > 0.05).
The effectiveness was indirectly demonstrated by its ability to alert to the presence of a pulmonary lesion that a doctor might have missed. While the proposed DLAD may produce more false positive results, its ability to identify overlooked lesions makes it a useful decision-support tool in clinical practice. For individual radiologists, it was 62, 46, 24, 10 and 21 CXR images with missed suspicious lesions that the assessed radiologists would re-evaluate and/or seek guidance from more experienced colleagues.