Augmented Reality: What is it and How is it Enhancing Healthcare Today?

By Mark D. Wiederhold & Brenda K. Wiederhold

While Virtual Reality (VR) has been a more established field of research for some time, Augmented Reality (AR) is a slightly newer field of research whose popularity, rate of use, and applicability to various uses has literally exploded with the increased usage of smartphones around the world. While readers are likely familiar with the term “apps” – applications downloaded to enhance the usability of a smartphone – the interpretation of AR may not be so clear cut. Questions include: What is AR? What does it encompass? And what are its applications?

What is Augmented Reality?

While VR aims to immerse the user in a purely virtual world, AR attempts to bring virtual elements to the real world. It is best described as an “AR Continuum” in which the real and virtual environments are placed at opposite ends with AR and augmented virtuality (AV) towards the middle.

AR can be defined as “the art of superimposing of computer generated content over a live view of the world” according to Dan Sung, a Pocket Lint blogger who writes on the subject. The viewer’s realtime view of either the real, surrounding environment or streaming video is fused with digital information. The paramount advantage of AR over VR is that it creates an altered or augmented reality without losing the benefits of the physical setting – touch, smell, hearing, taste, and visual contact with other people.

How does it Work?

Combining real and digital elements can be done in one of three ways: using a Head-Mounted Display (HMD), placing the visual information close in front of the user’s point of view; using handheld devices, most commonly smartphones and tablets; and computer-generated overlay that is placed directly on real objects using projects or devices known as Spatial Displays. Each method has distinct advantages and disadvantages.

While HMDs have been thoroughly researched and developed, leave the user’s hands free and the entire visual field is augmented, the hardware can be distracting, bulky, and expensive. Handhelds are convenient, cheap and commonly carried by the majority of users, yet require the user to give up the use of a hand and the field of view is limited and framed. Viewing Spatial Displays means more than one user is exposed to the same picture and participants don’t need to hold or wear anything, but the projections only work with specifically designed environments and again, set-up costs and specialized hardware can be costly and daunting.

Although enhanced visualization is most commonly brought to mind when discussing AR, it is important to remember AR can hypothetically be used to enhance any of the senses, for example, with haptics feedback to alter the sense of touch. Still, the field of augmented vision is currently experiencing the most growth and potential for applications in various fields.

Augmented Reality In Medicine

Austrian mathematician Steinhaus first described an augmented system to visualize a piece of metal embedded within tissue in 1938 when medical practitioners began envisioning ways in which real-time x-ray vision would make it possible to visualize patient data within the same space as the patient. In 1968 Ivan Sutherland described the first HMD and in the ‘90s research into medical applications of AR began in earnest.

Essentially, the clinical applications of AR in medicine are made up of two types of systems. As Dr. Andreas Pusch states, “On the one hand, there are systems in which synthetic multimodal head-up information is overlaid on tracked real objects (e.g., patient history and rehabilitation progress data, or specific information attached to certain body parts) while on the other hand, others permit direct guidance and training of surgeons and therapists by means of interactive co-located real-time simulations (e.g., visuo-haptic remote navigation of inspection or surgery equipment, annotations of regions to operate on, 2- D/3-D visualizations of hidden organs and structures).” For an in-depth look at Pusch’s full article, see page Features: Manipulative Augmented Virtuality