- Augmented and virtual reality are still new, but they may soon write a new chapter in health care.
- We are now able to display a level of photorealism that is difficult to discern from reality. [1].
Fig. 1: VR vs.AR vs.MR
References: author
Augmented Reality
- Defined as an enhanced version of reality created by the use of technology to add digital information on an image of something.
- AR is used in apps for smartphones and tablets.
- For example:AR apps use your phone's camera to show you a view of the real world in front of you, then put a layer of information, including text and/or images, on top of that view.
- Apps can use AR for fun, such as the game Pokémon GO, or for information, such as the app Layar.
Virtual Reality
Mixed Reality is also called as hybrid reality
- It is the merger of real and virtual environments in order to create new environments and visuals.
- In that new environment, both physical and digital entities exist together, interacting in real time.
- It means a new imagery is placed inside a real space in such a manner that the new imagery can interact to a degree, with the real world as you know it.
- The distinguishing factor of MR is that the artificial content and the real or physical world content can interact with one another in real time [3].
Fig. 2: Augmented reality VS.Virtual Reality Vs,Mixed Reality
References: https://medium.com/@shivsoni377/the-difference-between-augmented-reality-virtual-reality-and-mixed-reality-a028bdd81f9d
Fig. 3: AR /VR/MR
References: Silva, J. N., Southworth, M., Raptis, C., & Silva, J. (2018). Emerging Applications of Virtual Reality in Cardiovascular Medicine. JACC: Basic to Translational Science, 3(3), 420–430. doi: 10.1016/j.jacbts.2017.11.009
Fig. 4: HeARt: Augmented and Virtual Reality (AR / VR) for Medical School Learning Tool Enhancement
References: https://www.youtube.com/watch?v=dWf7oEwZKbs
Fig. 5: VR in operating theatre
References: https://innovationorigins.com/philips-brings-microsofts-hololens-into-the-surgeons-operating-room/
A- The steps of deriving 3D anatomical model from DICOM data:
Fig. 6: Steps of generating 3d print models/VR/AR images from 2D images
References: author
Fig. 7: The steps required for 3D anatomical modeling from DICOM data
References: Wake, N., Alexander, A.E., Christensen, A.M. et al. 3D Print Med (2019) 5: 17. https://doi.org/10.1186/s41205-019-0054-y
- Example of creating patient-specific anatomical models for AR/VR, for pelvic fracture :
Fig. 8: a Coronal CT image showing threhsolded right pelvic bones, showing similar colors for the pubis, ischium, and femur. b Coronal CT image showing splitting of the pelvis (blue) from the femur (black). c 3D computer model showing the pubis (white) and ischium (yellow). d Photograph of 3D printed model or AR/VR
References: Wake, N., Alexander, A.E., Christensen, A.M. et al. 3D Print Med (2019) 5: 17. https://doi.org/10.1186/s41205-019-0054-y
Fig. 21: Overview of how D3D AR/VR system could be used to evaluate how a tumor at multiple time points.
References: https://www.intechopen.com/books/state-of-the-art-virtual-reality-and-augmented-reality-knowhow/augmented-reality-and-virtual-reality-initial-successes-in-diagnostic-radiology
B-Image processing for AR/VR:
To optimize visualization, the D3D software suite must provide the capability for visualizing medical imagery rendered in a true 3 dimensional representation.[4].
Images processing can be summarized in fig 9.
Fig. 9: Summary of Imaging processing for AR/VR
References: Author
Fig. 10: 3D segmentation process
References: Douglas, D. B., Venets, D., Wilke, C., Gibson, D., Lance, Liotta,.Douglas, R. (2018, March 8). Augmented Reality and Virtual Reality: Initial Successes in Diagnostic Radiology.
Fig. 11: 3d filtering Algorithm
References: Douglas, D. B., Venets, D., Wilke, C., Gibson, D., Lance, Liotta,.Douglas, R. (2018, March 8). Augmented Reality and Virtual Reality: Initial Successes in Diagnostic Radiology.
III-3D rendering engine basic concept and geometry is summarized in figs: 12-14.
Fig. 12: 3D rendering engine
References: Douglas, D. B., Venets, D., Wilke, C., Gibson, D., Lance, Liotta,.Douglas, R. (2018, March 8). Augmented Reality and Virtual Reality: Initial Successes in Diagnostic Radiology.
Fig. 13: Figure illustrates left eye viewing perspective (LEVP) and right eye viewing perspective (REVP). The angles from the LEVP through the volume of interest can converge to a convergence point “c” to enhance visualization of a small volume of interest.
References: uglas, D. B., Venets, D., Wilke, C., Gibson, D., Lance, Liotta,.Douglas, R. (2018, March 8). Augmented Reality and Virtual Reality: Initial Successes in Diagnostic Radiology.
Fig. 14: Five examples of augmented reality/virtual reality viewing options with the D3D technology.
References: uglas, D. B., Venets, D., Wilke, C., Gibson, D., Lance, Liotta,.Douglas, R. (2018, March 8). Augmented Reality and Virtual Reality: Initial Successes in Diagnostic Radiology.
Advantages of Utilizing AR/VR technologies:
Fig. 15: Advantages of Utilizing AR/VR over Traditional Medical Imaging Techniques
References: Author