Keywords:
Computer applications, Management, Lung, Neural networks, CT, RIS, Computer Applications-General, Cost-effectiveness, Statistics, Embolism / Thrombosis, Quality assurance, Economics
Authors:
E. Sjöblom1, C. Lundström1, M. Andersson1, N. Carius2, J. Taghia3; 1Linköping/SE, 2Ljungsbro/SE, 3Uppsala/SE
DOI:
10.1594/ecr2018/C-0108
Results
Report classification: The accuracy of the report classifier was evaluated on the test set and reached an accuracy of 98% and an F1-score of 0.93.
Fig. 7 show examples of reports classified correctly and incorrectly.
The classifier mainly struggles with reports where there is an addendum contradicting the original statement and reports that are very long (often due to reports covering multiple reasons for examination).
Confusion matrix: To estimate the transition-probabilities for the classifier we compute the confusion matrix on the development set (keeping the test set untouched).
|
Û=0 |
Û=1 |
U=0 |
2956 |
51 |
U=1 |
43 |
519 |
From which the transition-probabilities can be estimated.
Fig. 8: Transition-probabillities computed on the development set.
And thus p1|1 = 0.923 and p1|0= 0.017.
Rate prediction: The prior on γ can be used to encode additional information that might be available about the expected rate of positive findings.
The prior regularizes the result and the shape can be treated as an application dependent hyper-parameter.
We will in the following assume an uninformative prior (s=f=1).
We evaluate the positive rate predictions on random subsets of the test data.
For each sample size (n= 50,
100,
300,
1000) we randomly draw examples from the test set,
classify them and compute a 95% credible interval for the observed rate of positive findings.
The computed interval is then translated according to Eq.
2 using the fixed transition-probabillites T,
computed from the development set.
For each subset we also report the fraction of reports that actually were positive as the True rate. The results are summarized below and visualized in Fig. 9 .
N
|
I95(γ)
|
I95(θ)
|
True rate
|
50
|
0.113-0.331
|
0.106-0.347
|
0.18
|
100
|
0.142-0.300
|
0.138-0.312
|
0.19
|
300
|
0.126-0.209
|
0.120-0.212
|
0.17
|
1000
|
0.134-0.179
|
0.129-0.178
|
0.16
|