In the management of radiology operations, it is important to understand the effectiveness of diagnostic pathways. One source of data to underpin managerial decision-making is the radiology report, but large-scale mining of such data is a challenge. The challenge comes from extracting actionable data from the free-text format that most radiology reports consist of [3]. We here study at the problem of quantifying the rate of positive findings in CT pulmonary embolism examinations. The number of examinations with positive findings versus the total number of...

Report data: Reports for CT pulmonary embolism examinations from a university hospital was collected (n=22305). The reports were categorized into five categories by two medical experts. The experts only had access to the clinical history and the report text, no images (Fig. 1). The categories (decided on by the medical experts) were: Embolism (n=3512) No embolism (n=16890) Previously known embolism/No increase (n=160) Uncertain (n=289) Not applicable (n=1454, these include reports for examinations that had been incorrectly marked as CT pulmonary embolism examinations) The dataset was...

Report classification: The accuracy of the report classifier was evaluated on the test set and reached an accuracy of 98% and an F1-score of 0.93. Fig. 7 showexamples of reports classified correctly and incorrectly. The classifier mainly struggles with reports where there is an addendum contradicting the original statement and reports that are very long (often due to reports covering multiple reasons for examination). Confusion matrix: To estimate the transition-probabilities for the classifier we compute the confusion matrix on the development set (keeping the test...

The proposed method can accurately estimate the rate of positive findings in a set of reports. The method takes into account that using any type of machine learning classifier to predict if a report includes a positive finding will introduce a source of error. It tries to mitigate this source of error by utilizing the confusion matrix of the classifier. The model can be adapted to different scenarios by injecting additional information into the shape of the prior on the observed rate of positive findings....

E. Sjöblom, Msc CSE Sectra AB Teknikringen20 SE-58330Linköping SWEDEN Phone: +4613235200 Fax: +4613212185 Email: [email protected]

[1] Lovins, J. B. (1968). Development of a stemming algorithm. Mech. Translat. & Comp. Linguistics, 11(1-2), 22-31. [2] Pedregosa et al. (2011), Scikit-learn: Machine Learning in Python, JMLR 12, pp. 2825-2830. [3] Pons et al.(2016) Natural Language Processing in Radiology: A Systematic Review. Radiology. [4] C. E. SHANNON (1948). A Mathematical Theory of Communication. The Bell System Technical Journal, Vol. 27, pp. 379–423, 623–656 [5] Downey, A. B. & Loukides, M. & Spencer, A. (eds.) (2013). Think Bayes. Sebastopol, California: O'Reilly Media.