Keywords:
Artificial Intelligence, Neuroradiology brain, CT, CAD, Computer Applications-Detection, diagnosis, Ischaemia / Infarction
Authors:
A. Ayobi, P. Chang, D. Chow, C. Filippi, S. Quenet, M. Tassy, Y. Chaibi
DOI:
10.26044/ecr2023/C-19206
Methods and materials
A retrospective, multicenter, multinational, multivendor and blinded study was conducted to evaluate the standalone performance of CINA-ASPECTS v1.4.2 (Avicenna.AI, La Ciotat, France), a CE-marked AI-based algorithm designed to detect signs of EIC on NCCTs and automatically compute the ASPECT score.
Data and Ground Truth (GT)
One hundred thirty-nine (139) NCCT images pertaining to patients with confirmed acute MCA and/or ICA occlusion were retrospectively collected. Two board-certified expert neuroradiologists proceeded with the visual assessment of the dataset to determine if there is EIC on each ASPECTS region. In case of discrepancy, a third board-certified neuroradiologist reviewed the NCCTs and the final GT (presence or absence of EIC within the 10 ASPECTS regions of each hemisphere) was established by majority agreement.
Standalone Performance
The comparison between the results automatically computed by CINA-ASPECTS and the GT was performed according to a confusion matrix that provided the number of true positives, true negatives, false positives and false negatives. Region-based sensitivity, specificity and Receiver-Operating-Characteristic (ROC) Area-Under-the-Curve (AUC) were calculated over all ASPECTS regions.
Multi-Reader-Multi-Case Study
Moreover, a multi-reader-multi-case (MRMC) study, using a subset of 40 NCCTs from the initial 139 scans, was performed to evaluate the effect of the software use on physicians’ interpretations. The MRMC study was assessed with three additional readers (radiologists different from the ones who established the GT), each of whom interpreted the exams first without software assistance and, after a washout period, with assistance. The readers used a 6-point confidence scale for the analysis of each ASPECTS region. Readers’ assessments were compared to the GT previously established. Improvement in reader performance was determined by computing the difference in reader’s ROC AUC with and without software assistance. The analysis was conducted following the Obuchowski-Rockette-DBM-MRMC methodology [12].