Eastern Europe is one of the worldwide regions most affected by tuberculosis, with Romania being one of the countries where the disease is widely spread [5].
The “Marius Nasta” Pneumology Institute initiated the “Organizing Programmes for Early Screening, Diagnosis and Treatment of Tuberculosis” project.
The project:
-
aims to help combat tuberculosis in Romania by early detection and constant treatment;
-
focuses on people with limited access to medical diagnosis or treatment;
-
total target: 75.010 individuals.
Four caravans containing X-ray machines and acting as mobile scanner points operated by a team formed by a driver, a hospital legal representative and a radiographer reached different regions of the country in order to reach the target population. A Computer-Aided Diagnostic (CAD) software application is fitted on each caravan and it is used as a screening tool.
Every X-ray performed on people was automatically processed by the CAD application, which attributed a score, representing the chance of tuberculosis being present in the image.
If the score was above a certain threshold, the medical team performed the Xpert (a rapid TB exam) [6] on the patient, who was notified about the results as soon as possible. Radiologists later analysed the x-rays as well, and people are notified of any other possible anomaly.
Deep learning models have become a popular backbone for radiology Computer-Aided Diagnostic software programs, with convolutional neural networks being the algorithms that perform best for multiple tasks, such as:
-
mammography cancer detection [7];
-
chest pathology detection [8];
-
bone anomaly detection [9].
Consequently, we set off to develop the CAD needed by the Tuberculosis Screening project by training our own convolutional network on a custom dataset and we plan to provide it for use if it shows good results.
We designed an architecture starting from the standard DenseNet121 [10] network and tried multiple enhancements in order to see which works best. The greatest improvement was adding the attention mechanism in [11], which helped the network understand better both coarse and fine grain information on the X-ray, which helps with recognizing lesions specific to tuberculosis.
The dataset is composed of radiographies from publicly available sources [12],[13],[14],[15], and anonymised data from Timisoara Clinical County Hospital, which were labeled using a combination of NLP algorithms and radiologists hand-labeling.
We split the dataset in train (80%), valid (10%) and test (10%). We trained each attention stage for 100 epochs, using the Adam optimizer and cycling the learning rate with a top value of 1e-3.
Validation Dataset
|
Best Threshold
|
Recall
|
Precision
|
F1
|
AUC
|
Timisoara County
|
60%
|
86%
|
87%
|
87%
|
93%
|
Montgomery + Shenzhen
|
53%
|
67%
|
97%
|
80%
|
90%
|
Padchest
|
39%
|
87%
|
78%
|
83%
|
90%
|
Table 1. Metrics used to understand model results. In order to check the model’s ability to generalize, all the metrics are generated by cross-validation (i.e. excluding one dataset in training and using it for validation)
In order to comprehend the outputs of the model and confirm that it works, we extracted attention maps and used them to understand which parts of the image activated the network.
Fig. 2: Chest X-ray showing nodular lesions of various densities, some with irregular borders, associated with the prominence of linear, interstitial markings and pleural thickening in the upper third of the right lung.
Fig. 3: Chest X-ray demonstrates inhomogeneous lesion in the apical region of the right lung; multiple infracentimetric nodules of various densities, some of them calcified, situated in the upper third of the right lung.
Fig. 4: Multiple infracentimetric nodular lesions with regular borders, of various densities, some of them calcified, associated with fibrotic lesions in both upper lungs. Multiple bilateral healed rib fractures.
Fig. 5: Multiple infracentimetric nodular lesions, most of them calcified, distributed in the upper half of both lungs, suggesting inactive fibrotic lesions.
Fig. 6: Multiple nodules of various dimensions and densities, associated with the prominence of interstitial markings in the upper third of the right lung.
We used this model to screen and score patients from the Tuberculosis Screening project, establishing an initial threshold of 60%, for which the recall is 90% on the test dataset.
Number of patients
|
Radiologist confirmed TB
|
Xpert tested (algorithm score > 60%)
|
Active TB confirmed by Xpert
|
1010
|
34
|
41
|
9
|
Using partial results on a batch consisting of the first 1010 people that participated in the project, we generated statistics in order to assess the viability of the algorithm and decide whether threshold adjustments should be made.