Current applications of AR/VR in Radiology:
1. Improve radiology training and education.
- AR is already being used to help some trainees “conceptualize complex anatomy,” through the process of converting DICOM images to 3D objects.
- VR can help a trainee practice how to react in certain situations such as contrast agent reactions.
- Another way these technologies can help with training students is by improving how professors give lectures, inparticular using VR when training interventional radiology students. [5]
Fig. 16: HoloLens app: Radiology with Holographic Augmentation (RadHA). To see the anatomy represented in a 3D way is more realistic and closer to what they actually encounter when in the operating room. The possibilities of superimposing patient-specific 3D medical objects onto a real-world background have generated new ideas for patient and trainee education, presurgical planning, and even intraoperative use.
References: Radiology, News, Education, Service - auntminnie.com. (n.d.).
Fig. 17: 3D kidney tumor model visualized a in AR using the HoloLens AR headset (Microsoft, Redmond, WA), b in VR using Syglass software (Syglass, IstoVisio, Inc., Morgantown, WV) in combination with the Oculus Rift (Facebook, Menlo Park, CA), and c in VR using the Sketchfab app (Sketchfab, New York, NY) and a smartphone device. Each structure is numbered so that the unfamiliar user can easily identify each individual structure: 1 – kidney, 2 –vein, 3 – artery, 4 – collecting system, 5 – renal tumor
References: Wake, N., Alexander, A.E., Christensen, A.M. et al. 3D Print Med (2019) 5: 17. https://doi.org/10.1186/s41205-019-0054-y
Fig. 18: Using Augmented Reality Applications to Visualize 3D Radiology Images: comminuted fracture of the femur .
References: https://pubs.rsna.org/doi/10.1148/radiol.2019182210?fbclid=IwAR2Baf5aE7Q8t7T0R82mxD5In4mm0rp_YrUVpkrNlg6A4Z_QdlAIYNBv0G0
Advantages:
- Comparable anatomy knowledge to those taught by traditional methods of cadaveric dissection and textbooks.
- Medical students had greater answering speeds in anatomy assessments, greater rates of satisfaction with the learning
methods, and a greater enthusiasm for their future careers when compared to conventional methods.
- New 3D VR headsets such as the Microsoft Hololens show potential for real-time pathology–radiology correlation and evaluation of gross pathology.[7]
2. Virtual reality proves a useful treatment tool for interventional radiologists . (New study by Wayne Monsky, a professor of radiology at the University of Washington)
Fig. 19: A futuristic image shows an interventional radiologist doing a procedure in a model patient's blood vessels with the VR rendering of the vessels they are seeing in their headset
References: https://www.dailymail.co.uk/health/article-6849401/New-VR-technology-lets-doctors-travel-patients-blood-vessels.html
-
Improving treatments using real-time 3D images from inside a patient’s vessels.
-
This technology will allow physicians to travel inside a patient’s body instead of relying solely on 2D, black-and-white images.
-
Using a CT angiography scan, a 3D-printed model was developed and a holographic image of blood vessels in a patient’s abdomen and pelvis.
-
The radiologists then guided catheters fitted with electromagnetic sensors through the 3D printed model while a tracking system projected the image from the catheter through a VR headset.[8]
Advantages:
- The mean time it took to reach the targeted vessels was less when using VR over fluoroscopy—VR took 17.6 seconds to reach the first vessel compared to 70.3 seconds using fluoroscopy in the 3D model and 171.2 seconds in a real-life setting.
- VR approach doesn’t only benefit radiologists themselves—who reported they felt more confident, precise and efficient when using VR—but extends that benefit to patients with poorer access to healthcare.[8]
3-Scope of Applying AR/VR in Oncology ImagingFig. 20: Scope of applying AR/VR in Oncology Imaging
References: Author
Fig. 21: Overview of how D3D AR/VR system could be used to evaluate how a tumor at multiple time points.
References: https://www.intechopen.com/books/state-of-the-art-virtual-reality-and-augmented-reality-knowhow/augmented-reality-and-virtual-reality-initial-successes-in-diagnostic-radiology
4. Help colleagues communicate with one another
Fig. 22: When you put on the VR goggles, it’s like you’re actually in the radiology suite. You can look up, down, left, right — it’s a full 360-degree view.
References: Virtual Reality Radiology (with VIDEO). (n.d.). Retrieved from https://makingofmedicine.virginia.edu/2018/03/13/into-the-or-in-vr/.
- Planning and executing complex surgical procedures require a comprehensive understanding of anatomic relationships,
- There are several commercially available simulators available. Three-dimensional learning environments, such as those available using VR technology, offer the potential to increase spatial representation and improve the planning of complex surgical tasks, such as those required of a neurosurgeon.”
Users can also share AR images remotely, allowing them to interact with the same images at the same time from different parts of the world.(video) thus used it as a teaching tool. [9]
5-To Help Ease Patient Anxiety.
- The virtual reality display of a patient’s anatomy can be utilized for preprocedural planning, ultimately affecting the way a patient’s procedure is performed.
- Simulating a radiology procedure with virtual reality can help a patient feel more comfortable.[14]
How A radiology room can look like with VR/AR?
Fig. 23: how radiology room can look like with VR/ AR?
References: https://www.slideshare.net/joaquimjorge94/virtual-relity-for-health-applications-siggraph-asia-2018
Let us take a look at the limitations of AR/ VR as a whole.
Fig. 25: challenges of VR/AR
References: Author
Fig. 24: Limitations of AR and VR
References: author