1.
Agreements on the lesion classification:
Intradevice agreements on the lesion classification observed in our study,
for individual radiologist and for the overall group are presented in Table 2.
Observed agreements were ranged from 63.6–82.6%,
with kappa values ranged from 0.47–0.74,
and agreements ranked as “Substantial” for two radiologists and “Moderate” for four radiologists (P < 0.001).
The overall radiologists observed agreement on lesion classification was 73.0%,
with a value kappa of 0.6 and ranked as “Moderate” (P < 0.001).
These values were evaluated for each lesion type and result are shown in Table 3.
All the observed agreements were ranged from 73.9–98.2%,
with high kappa values,
ranged from 0.50–0.89,
and all ranked as “Moderate” or “Almost Perfect”.
The higher agreement was on hemorrhagic lesions,
and the lower on chronic lesions.
Nevertheless,
when chronic and acute ischemic lesions were combined together into a category “Cerebral Infarction”,
high agreements were noted,
with observed agreement of 81%,
kappa of 0.6 and ranked as “Substantial”.
In the interpretation process,
radiologists were asked to evaluate cases interpreted as ischemic lesions,
for the presence or not of possible contraindications to receive the tPA treatment; for this variable named “Ischemic without contraindications”,
“Moderate” agreements were observed,
with observed agreement of 75% and kappa of 0.43.
Intradevice agreement on the lesion classification by reading system,
for individual lesions types and for the overall classification are presented in Table 4.
The higher intradevice agreements were observed for hemorrhagic lesions; agreements were ranked as “Almost Perfect” in both Medical-IMPAX and TABLET-XERO reading systems,
with kappa values of 0.90 and 0.82,
respectively.
The lower intradevice agreement was observed for chronic lesion using the Medical-IMPAX,
with a kappa of 0.37,
ranked as “Fair” agreement.
Nevertheless,
the rest of agreements were ranked as “Moderate” for both reading systems.
For classification of “Cerebral Infarction” (acute or chronic),
observed agreements were ranked as “Moderate” in both reading systems,
with kappa values of 0.58.
2.
Agreements on the ASPECTS classification:
The observed interdevice agreement on the dichotomized-ASPECTS classification (1-6;7-10),
for the overall group of radiologists (Table 5) was ranked as “Substantial”,
with a kappa value of 0.65,
and a high-observed agreement of 92.3%.
For grouped ASPECTS categories (i.e.,
1-3,
4-7,
8-9,
and 10),
the observed agreements were ranged from 80.7–98.6%,
and kappa values were ranged from 0.43–0.66; with “Substantial” agreements for ASPECTS 1-3 and ASPECTS 10 categories,
and “Moderate” agreements for ASPECTS 4-7 and ASPECTS 8-9 categories.
3.
Reading time:
The reading time for interpretations on the Medical-IMPAX and Tablet-XERO were 126 s and 123 s,
respectively,
with no statistical significant difference (P = 0.566).
The difference time to claim equivalence was 13.7 s (P < 0.05).
4.
Discussion:
High interdevice agreement on the lesion classification (i.e.,
agreement for radiologist when reading the same case with Medical-IMPAX and Tablet-XERO systems) were observed,
all ranked as “Moderate” or “Substantial”.
Intradevice agreement on the lesion classification by reading system were all ranked as “Moderate” agreement for the combined classification,
for both Medical-IMPAX and Tablet-XERO systems.
For individual lesion type agreements (i.e.,
hemorrhagic lesion,
ischemic lesion,
chronic lesion,
and without acute lesion) the marginal kappa values,
in general,
were lower than those for the combined classification were.
This is a well-described effect by Feinstein and Cicchetti when variables are separated in individual categories.
Although the marginal kappa values were low for chronic lesion,
very high agreements were noted on hemorrhagic lesion (ranked as “Almost Perfect” or “Substantial).
These results on hemorrhagic lesion in our study were as well in line with the results of some other studies where high agreements and accuracy were found for hemorrhagic lesions.
Low agreements,
ranked as "Fair”,
were noted for chronic lesions on Medical-IMPAX.
This effect may be due to a limitation of our study,
as initially,
for patients with multiple findings,
as acute or chronic events,
radiologists doubt to select a lesion type.
For readings with the TABLET-XERO system,
it was defined to classify these patients as acute (instead of chronic),
which increased their agreements with respect to the Medical-IMPAX system that was the first used.
Very high agreements were noted on interdevice agreement on the dichotomized-ASPECTS classification (i.e.,
agreement for radiologist when reading the same case with Medical-IMPAX and Tablet-XERO systems),
with agreements ranked as “Almost Perfect” or “Substantial” for induvial radiologist or for the overall group.
This is a relevant fact suggesting that the Medical-IMPAX and Tablet-XERO systems may be interchangeable,
without loss in reliability.
Intradevice agreement on the dichotomized-ASPECTS classification were all ranked as “Moderate” agreement,
regardless of the reading system.
One limitation of this study is that only cases classified as ischemic lesion,
with no contraindications to receive the tPA treatment,
were evaluated for the confidence of the presence of HIA.
Therefore,
the number of cases with scores assigned by the six radiologists on the two reading systems (i.e.,
12 scores assigned by case),
was every low to achieve a good analysis of this variable.
Another limitation of our study is due to the illumination conditions.
Readings using the Medical-IMPAX system were performed in a diagnostic reading stations with ambient lighting levels controlled according to the AAPM TG18 recommendations (i.e.,
15-60 lux),
in contrast,
readings using the TABLET-XERO system were performed without controlling ambient lighting levels.
Nevertheless,
this situation is more realistic for a telestroke system in which a radiologists is asked to read the CT examination as soon as possible wherever he is.