Aims and objectives
In imaging research and development a rigorous evaluation of the reproducibility of diagnostic tests before dissemination into clinical practice is essential.
Radiological observer performance studies are typically conducted by having observers read cases at a fixed location on a PACS (Picture Archiving and Communication System) workstation or DICOM (Digital Imaging and Communications in Medicine) viewer installed on a standalone PC while recording their measurements either into an electronic database or with pen and paper.
This practice poses logistical challenges,
and a concern is that the...
Methods and materials
Development and functionality
The app was developed as a cloud-based system [6] using the following open-source tools:
The PHP programming language handles server-side functions.
Data is stored in a mySQL database
The client-side presentation layer uses HTML5 and javascript thus ensuring compatibility with all major internet browsers.
Study cases are prepared by de-identifying and exporting DICOM studies to JPEG files (retaining native image resolution) and adding relevant DICOM tags to the mySQL database.
The app can handle multiple modalities including but not limited to: plain...
Results
The app was tested on MAC and PC laptop/desktop computers using browsers: Chrome,
Firefox,
Safari,
Internet explorer,
and Opera with full functionality on screen resolutions from 800x600 to 4096×2160 pixels.
None of the three invited observers reported problems accessing the app and were able to complete the case readings without any further instructions than the one that came in the invitational e-mail and the app (Fig.
3).
TABLE 1
Observer
Total reading time (300 cases)
Cases/hour
Observer 1
34 minutes 25 seconds
523/hour
Observer 2...
Conclusion
This feasibility study shows that the developed app allows invited observers to be scattered in localization where case readings can be separated in time and not necessarily have to be completed all at once.
Thus,
using this web-based tool it is possible with relative ease to efficiently conduct multicenter observer performance studies with potential significant savings in time under reasonably realistic conditions with a sufficiently high number of observers/cases.
In the future the Content Mangement System of the app will be further developed to allow...
Personal information
Jens Borgbjerg MD
Department of Radiology
Aalborg University Hospital
Aalborg
Denmark
References
1: Di Leo G.
Measurements in radiology: the need for high reproducibility.
Pediatr Radiol.
2015 Jan;45(1):32-4.
doi: 10.1007/s00247-014-3081-2.
Epub 2015
Jan 1.
PubMed PMID: 25552389.
2: Bankier AA,
Levine D,
Halpern EF,
Kressel HY.
Consensus interpretation in
imaging research: is there a better way? Radiology.
2010 Oct;257(1):14-7.
doi:
10.1148/radiol.10100252.
PubMed PMID: 20851935.
3: Sadatsafavi M,
Najafzadeh M,
Lynd L,
Marra C.
Reliability studies of
diagnostic tests are not using enough observers for robust estimation of
interobserver agreement: a simulation study.
J Clin Epidemiol.
2008
Jul;61(7):722-7....
