Virtual Reality and Visualization
Virtual reality refers to a set of techniques in which one interacts with a synthetic (“virtual”) environment that exists solely in the computer. In the typical conception of virtual reality, the representation of the synthetic environment is fed fairly directly to the eyes , ears , and possibly hands. The actions of the user in the environment are translated directly from typical physical activities. There is a continuum of realizations of this ideal, involving compromises on these input/output modalities. At one end of the continuum, which we call a “complete virtual reality simulation,” the participant is immersed in a virtual world that fully replicates at least three sensory inputs—vision, hearing, and touch (the last is more technically known as a haptic/kinesthetic system)—and allows complete physical interaction with the world. At the other extreme of the continuum is a screen-based simulator, which generates a limited virtual world, but it restricts its output to a screen display and provides interaction with the virtual world only through a pointing device. The screen-based simulator provides an interface to the human sensory system that is very far from physical reality, whereas a complete virtual reality simulation may be, in its most advanced form, nearly indistinguishable from the real world. A “partial” virtual reality simulator would replicate fewer senses (or less complete replication, such as a 3D visual representation on a 2D screen) and/or could restrict physical interaction with the world. Finally, one can imagine hybrids of realistic simulators and virtual reality simulators in which the virtual reality representation is overlaid onto a real physical environment.
A variant of virtual reality is often called “Visualization” This involves presentation of 3D structures (such as anatomy or molecular structure) in ways that maximize learning. This can involve true stereoscopic viewing of such structures using special projectors and glasses or special head-mounted displays with a separate view for each eye. Other forms of visualization present only a perspective image on a two-dimensional screen, but with such high resolution and with such extensive means of manipulating the images (such as rotating it, zooming, etc.) that any part can be viewed in detail (See a video of how this might work. Visualization is not only a simulation technique for learning, but also can be used as part of patient care (to plan complex procedures) and to view research data (turning numbers into pictures).
Watch a video of visualization in action… Virtual reality is often used to create advanced part-task trainers for complex medical procedures such as minimally-invasive surgery (e.g. laparoscopic surgery) or catheter-based “endovascular” procedures such as coronary angiography, angioplasty, and stent placement. Virtual reality lends itself to these procedures because the clinician in the real world already is interacting with an image on a screen (from a camera in laparoscopy; from the X-ray fluoroscopy in the “cath-lab”) using artificial extensions of the hands (laparoscopic surgical instruments and long, thin catheters in the blood vessels in the two examples).
Creating a complete virtual reality replication of the entire patient and treatment setting is very complex as it would require the following:
- A complete computer model of the patient, the environment, and the function of every object in the environment that could be utilized (e.g., monitoring devices, carts).
- A means of tracking visual, audio, and touch fields of the user to determine what is to be displayed and to identify what physical actions are being performed.
- Appropriate display hardware for every sensory modality and appropriate input hardware for each action pathway (e.g., touch, speech).
- Hardware to compute all the models, to conduct the tracking, and to produce all the outputs to the display hardware in real time.
Virtual reality is a rapidly developing field. There is intense interest in virtual reality in a number of domains, particularly the military and entertainment. Although the potential of this approach is very exciting virtual
