countries

Virtual Reality – The Game Changer in Radiology?

What is Virtual Reality?[1]

Virtual reality (VR) is a technology that enables users to immerse themselves in a computer-generated, artificial environment, and to interact with it in real-time.
VR’s history dates back to the 1960s, when computer scientist Ivan Sutherland created an early prototype of a head-mounted display (HMD) capable of displaying basic wireframe graphics. Since then, VR technology has seen significant advancements, including the development of more sophisticated HMDs, motion tracking, and haptic feedback systems. Today, a variety of terms refer to the creation of digital realities, such as VR, augmented reality (AR) and mixed reality (MR). But how do they differ?

While AR, VR and MR are often used interchangeably, they are actually distinct technologies:

  • AR enhances the real world by adding digital elements to it, while VR creates a fully immersive digital environment.
  • MR merges aspects of both AR and VR, overlaying digital objects onto the real world while also allowing interaction with these virtual elements.
  • In VR, unlike in AR, there is no extension of reality with additional information layers. Instead, VR involves leaving the real world and using virtual reality glasses to immerse oneself fully into a separate, digital universe.

How does it work?[2]

VR creates a comprehensive system that replaces our natural “real-world” environment with an immersive simulation. Based on a combination of hardware and software, VR solutions provide a computer-generated alternative to our real-world view.

To produce an immersive and realistic experience, VR systems use a combination of infrared LEDs, motion sensors, cameras and screens. These components enable the headset to collect relevant information and present it to the human eye. Some solutions even incorporate accessories such as haptic feedback systems and spatial audio to enhance immersion.

True virtual reality doesn’t just fill your field of view with new information – it creates a simulated environment with which you can effectively interact. Using AI, sensors, and algorithms, VR headsets align the digital world with the real world, tricking your brain into believing you are in a new environment.

VR experiences typically rely on various types of hardware and software. These components have become increasingly advanced and sophisticated:

  • VR Headsets: These head-mounted wearable devices are crucial for many VR experiences. They block out the outside world and display specific information related to the VR simulation.
  • Lenses and Screens: Inside VR headsets, stereoscopic lenses between an LED screen and the eyes distort images to appear three-dimensional. The headset shows two images, one for each eye, while infrared cameras adjust the light, and trackers shift screen content as the user moves their head. Some headsets even track eye movement for foveated rendering.
  • Latency: Field of view and latency are vital for perceiving distance and spatial depth in both real and virtual environments. Humans can perceive an angle of 200 to 220 degrees, divided into monocular and binocular visual fields. Modern VR headsets aim to generate these areas accurately.
  • Frame Rate: The human eye captures around 1,000 images per second, but the optic nerve processes information at a slower rate. In VR, the ideal frame rate is typically 90 FPS, higher than cinema films but slower than everyday life in a busy city.
  • Position Tracking: Position tracking is essential for a successful VR experience. Popular systems include 6 degrees of freedom allowing users to move freely in space and perform gestures like nodding and moving in various directions. In contrast, 3 degrees of freedom only allows head movement to the right and left, and upwards and downwards.
  • VR Controllers: Many headsets come with controllers, while others use sensors for hands-free control of the VR environment.

Where is VR being used?[3]

VR is being utilized in various industries and for numerous applications, such as:

  • Automotive Industry: VR enables engineers and designers to easily experiment with the design and construction of vehicles before creating costly prototypes. Companies like Honda, BMW, and Jaguar Land Rover have been using VR for years to conduct design and engineering reviews, especially during the pandemic, when alternatives to traditional clay models were needed. VR is saving the automotive industry millions in man-hours and material costs by reducing the number of prototypes required per vehicle line and by allowing for a global workforce to collaborate during the design process.
  • Retail: VR retail experiences and body-scanning technology allow customers to try clothes on virtually, to see how they would look. This not only saves shoppers’ time, but also promotes sustainability by enabling customers to determine whether an item fits their shape and size before purchasing, and thus reducing the environmental impact of production and shipping. Brands like Ralph Lauren and Gucci have collaborated with VR Studio Emperia to create bespoke virtual store environments, so that high-end customers could explore their collections.

VR is present in many other fields, including education, entertainment, architecture, art and design, and even sports. But what about healthcare?

VR in Healthcare

The impact of VR on healthcare is also significant. In 2024, the VR in healthcare market was worth just over four billion U.S. dollars globally. That same market is foreseen to increase to over 13 billion U.S. dollars by 2029, with a forecast CAGR of 27.3%.[4] This development is incredibly fast, and is comparable to the growth of AI in medical imaging.

Healthcare professionals can use VR to prepare more efficiently for operating theater scenarios—whether as junior doctors explaining diagnoses and treatment plans, or as orthopedic surgeons performing surgeries.

Innovative companies like Brainlab are already investing in this technology. Brainlab has developed a Mixed Reality Viewer which turns patient images into lifelike 3-D visuals, allowing doctors to walk around and zoom in on specific areas of a patient’s anatomy. This tool is valuable for colleagues discussing a procedure, for instance, or for a doctor consulting with a patient.

Other companies enable surgeons to handle medical devices in VR and practice surgery on virtual bodies, making them more familiar and more proficient with new devices. Additionally, companies like Altoida are leveraging XR and machine learning to develop digital biomarkers for neurological diseases, such as Alzheimer’s disease. This technology platform offers earlier diagnosis, facilitates clinical trial participation, and enables professionals to monitor disease progression and treatment response more precisely.

In November 2021, the FDA approved EaseVRx for prescription use to reduce pain in adult patients. The system alleviates chronic pains through cognitive behavioral therapy and other behavioral principles, including deep relaxation, attention-shifting, and interoceptive awareness. VR has even been used as a form of pain relief for burns.

VR in Radiology [5]

Radiology has long been essential for surgeons, who rely on medical imaging to make diagnoses and plan precise procedures. VR and AR have now been introduced alongside medical imaging, to assist in procedural preparation. Javaid et al. see significant potential for VR to positively impact radiology.

Algorithms can convert CT data into a 3-D map, enabling doctors and other medical professionals to explore and manipulate images for various applications. Radiologists can resort to this technology for procedural planning, in-process guidance, detection and diagnosis. Additionally, VR has proven highly beneficial to train radiologists.[6]

Fierce Biotech recently reported a partnership between VR developer Surgical Theater and medical technology giant Medtronic. They aim to integrate AR with Medtronic’s surgical navigation platform to address aneurysms, tumors, and other conditions. Thanks to this technology, surgeons can test different approaches prior to surgery, and also benefit from live, 3-D renderings provided by the system, which are displayed above the surgical site during the actual procedure.

A Safe Way to Train New Doctors
Medical students can use VR and AR technologies as part of their training, to practice procedures more frequently. Vanderbilt University, for instance, uses a simulator which provides trainees with the physical sensations related to using instruments and feeling body tissue, while performing virtual surgery on a virtual patient. The VR system tracks and analyzes their progress in real-time.

A UCLA study showed that medical students who used VR tools for specific procedures demonstrated greater speed and accuracy. Additionally, as VR is not tied to a physical location, it offers opportunities for long-distance training, including for students in under-resourced areas and who have limited access to qualified teaching staff. During the pandemic, students were able to train while maintaining social distancing, using VR simulations.

VR and AR can also enhance radiology training. For instance, VR headsets can be of great value to students learning to examine medical images in more depth, by enabling them to view these images and interact with them in immersive 3-D.

Interventional Radiology
AR and mixed reality are exciting developments for interventional radiology, which uses medical imaging to guide real-time procedures such as biopsies, angioplasties and stenting. Interventional radiology is often chosen over surgical alternatives because it can be less costly and less risky, and can sometimes be performed without requiring a hospital stay. Interventional radiology, if combined with these new technologies, could become even more effective.

“With augmented or mixed reality, medical imaging can be more readily accessible or displayed in actual three-dimensional space during procedures to enhance guidance at times when this information is most needed,”[7] noted the authors of a study on the role of AR and MR in interventional radiology.

Having a detailed, multi-dimensional view of vascular anatomy can be extremely helpful when guiding a needle or catheter within a patient’s body.

VR Adoption in Radiology

Given the time it took to make computers ready for the mass market, it will also take some time for VR to become more popular, although many industries already benefit from its usage.

Patience and continuous investment in research and development will be necessary for VR to become a standard tool in radiology. However, the benefits and advancements we previously discussed already suggest that VR will not just be a standard tool – it will truly be a game-changer in this field.

The Dedalus Example: VR in Medical Imaging/Medical Imaging in VR: the Example of Dedalus

At RSNA 2023, we introduced our vision of how VR can shape the role of radiologists:

https://dedalusspa.sharepoint.com/:v:/s/DIITProductManagement/ETdv1u_E93VDkCLOo2FXjIABGv9Ie9TdywtyHvjwHdst-w?e=sWFZ84

AR in Radiology

Recent advances in technology have led to AR hardware being increasingly portable and thus more widely available for use. The main types of AR devices include see-through head-mounted displays, mobile-based devices, and projection-based devices. Head-mounted displays allow users to perceive a 3D object by passing light from micro projectors onto a waveguide, which is then projected back to the wearer’s eye.[8]

Like VR, AR can be used to train future radiologists, as well as to engage with patients. Early AR research revealed increased student motivation for learning, interaction, and subject learning. As the information is placed in a real-world context, the success of AR as part of the radiology curriculum is considered to be related to the unique nature of the experience, and to the retained ability for interaction.[9]

In addition, potential cost savings and time gains related to the independence of AR educational tools from any specific timeframe or place have also been highlighted.[10]

VR has also been tested with the Google Glasses HoloLens, not only in education but also in interventional radiology. The glasses can provide real-time guidance, assisting radiologists in precisely navigating catheters and devices during interventions such as angioplasty or stent placement.[11]

Current challenges
Despite promising advancements, AR technology in interventional radiology still faces significant limitations in accuracy, usability, and availability. Stabilizing AR images is challenging due to movement from both practitioners and patients. Current AR systems often lag between real-world movements and digital overlays. For instance, the HoloLens has an accuracy margin of about 1 cm, which is insufficient for complex procedures. Users also face issues such as visual focus and fatigue, as maintaining stereoscopic vision over long periods is difficult due to varying pupil distances. Additionally, advanced AR systems like the latest Microsoft HoloLens are expensive, hindering widespread adoption.

The British Society of Interventional Radiology envisions a promising future for AR as the technology continues to advance.[12]Elsakka et al. also recognize AR’s potential to revolutionize interventional radiology. The development of deep learning techniques, especially for medical imaging segmentation, could address current accuracy and precision issues.[13] Beyond technical feasibility and maturity, the acceptance of this innovative technology by radiologists is crucial for its success.

Expert Feedback

We were curious about the usage of VR/AR in Radiology and have asked a few questions to DACH- and Belgium-based radiologists to learn more.
The results of our survey, conducted among radiologists regarding VR and AR, were rather mixed but positive overall concerning the potential of these technologies in radiology.

Familiarity with VR/AR Technologies
Most respondents view VR/AR as promising for the future of radiology, particularly in surgical and interventional applications, as well as for educational purposes. Some of the respondents have already tested VR/AR in product demonstrations or experimental settings. While the technology is not yet part of their routine workflow, it could prove useful in complex cases, especially to improve assessments of pathologies and spatial relationships.

Use and Experience with VR/AR
Practical use of VR/AR among radiologists has for now been limited. A few have used VR for medical student training purposes or in test environments, for instance for evaluating high-resolution X-ray images. Reactions have been mixed, with students responding positively to VR/AR in simulations while remaining critical regarding the usability of the technology. Professionals find that VR headsets still require improvement, particularly when it comes to resolution, user interface and comfort of use. In addition, one radiologist noted that the weight of the headset became uncomfortable after prolonged use, and that external factors distorted the image quality.

Benefits and Challenges
Potential future benefits of VR/AR in radiology are foreseen mainly in diagnostics, interventions and training. VR could provide users with a more effective view of anatomical structures and pathologies. One respondent also highlighted the potential of enriching a virtual workstation with an optimized VR headset, which would allow for better integration of imaging and voice control. Challenges however still remain – the technology is currently cumbersome, concerns about data privacy persist (particularly with cloud solutions), and practical challenges remain to be solved such as headset weight, interaction for glasses wearers, and costs. Additionally, integrating VR into clinical workflows such as ultrasound procedures has proven difficult due to the necessarily direct interaction with patients and colleagues.

The Future of VR/AR in Radiology
Opinions vary as to when VR/AR will play a more significant role in radiology. Some experts believe it will become prominent within the next 3 to 10 years, while others are more skeptical, doubting its importance in radiology practice. In future, VR/AR is expected to enhance diagnostics and interventions, and to play a role in multidisciplinary case discussions.

Investment and Training
Although according to some radiologists, the current focus remains on other technologies including 3D printing, some hospitals are open to investing in VR/AR technologies. Training programs or professional development courses on VR/AR are however not yet offered.

In short, the survey shows that while VR/AR holds significant potential in radiology, several technical and organizational hurdles must be overcome before it can be widely integrated into everyday practice.

[1] Cf. Team Viewer.

[2] Cf. Today Digital Ltd (2023).

[3] Cf. Thompson (2024).

[4] Cf. Statista (2024).

[5] Cf. Panner (2022).

[6] Cf. Javaid et al. (2022).

[7] Panner (2022).

[8] Cf.  Medical Professionals 2024.

[9] Cf.  Medical Professionals 2024.

[10] Cf.  Raith et al. (2022).

[11] Cf. British Society of Interventional Radiology (2024).

[12] Cf. British Society of Interventional Radiology (2024).

[13] Cf. Elsakka et al. (2023).

List of references

Please open in latest version of Chrome, Firefox, Safari browser for best experience or update your browser.

Update Browser