Letter from the Secretary General and Editor-in-Chief

Dear Reader,

Augmented reality (AR) allows interactivity of both Virtual Reality and real-world elements. Some users wear special glasses that allow them to see the world as it is, as well as computer-generated objects that appear as display supplements to the real-world elements, as in the augmented optics or endoscopes in AR-assisted surgery. Other users may be in a specially designed room featuring image projections on objects, as in a factory in which assembly instructions are superimposed on component parts. Still others may use GPS-tracked smartphones with cameras as the input device, such as to take virtual tours of buildings, with information about office space for rent in them available in real time as the person arrives at that location.

But with so much information coming at us these days, what is the real benefit of AR? If texting while driving is such a distraction that it is banned in many U.S. states, will drivers using AR systems designed to improve safety experience distractions that will cause the accidents they are designed to prevent? Or is AR better used for entertainment, as shoppers use a smartphone app of a virtual pop-up store in different locations to “try on” different outfits and post them to Facebook?

While AR applications were developed as early as the 1960s, the cost of systems and the expertise needed to use them has limited their adoption in the past. Now, thanks to the ubiquitous smartphone, developers have a low cost of entry for this type of AR. Thanks to social networking, crowd-sourced content posted to “the cloud” creates large databases that developers can call upon to populate the fields in their applications. And thanks to the pervasiveness of computing in modern life, smartphone users have come to expect the ability to move about in the real world and simultaneously view an information overlay that seamlessly integrates data from a number of sources.

Among the first applications to show benefit were those in medicine. A recent review concluded that, “After two decades of research on medical AR the basic concepts seem to be well understood and the enabling technologies are now enough advanced to meet the basic requirements for a number of medical applications … A perfect medical AR user interface would be integrated in such a way that the user would not feel its existence, while taking full advantage of additional in situ information it provides … The AR window and video-see-through HMD systems still need hardware and software improvement in order to satisfy the requirements of operating physicians … medical AR will be one of its first killer applications, saving lives of many future patients.”

While medical uses of AR may set the highest bar for development, related uses are coming online. For example, for medical training, for which VR has long been used, the advantage of AR over VR systems is that they offer realistic haptic feedback while providing objective assessment of skill. In addition to helping adult learners such as physicians, AR may also be used to help children to learn. A recent study used AR in combination with three types of physical exercise to enhance non-memorized learning, and showed that physical exercise did not detract from ability to complete cognitive tasks.

In addition, the authors of a review of AR-assisted learning concluded, “The students we work with are already using their cell phones seamlessly to communicate and share information with their peers throughout the day. The findings from this study emphasize how engaged students become simply by using similar tools to learn.”

AR may also be used for different types of therapy. “TheraMem is a novel system which combines Augmented Reality, a simple computer game, and spatial-visual decoupling of the user’s hands for use in post-stroke rehabilitation. The system is considered to be mature and usable in the early stages of stroke recovery.” Another system still in development uses AR to help patients conquer fear of spiders and cockroaches, using a camera to put the patient in a real location and realistic sounds such as a spray can of insecticide or a squishing sound when a pest is killed.

Tracking an individual’s whereabouts opens the door for many uses of AR. For example, a museum education system uses a markerless pose estimation algorithm to guide a tourist to superimpose a displayed AR painting, with information that enhances the tourist’s understanding of the creative process, on the corresponding real-world painting. In the homes of people who have set up a home computer network, a program for the Android platform could track the WiFi signal of an individual’s smartphone to determine her location in the home. Outdoor tracking is more of a challenge, since GPS does not have the precision and refresh rate required, but researchers are working on methods such as multi-sensor fusion that integrates inertial and vision tracking to meet those requirements. A combination of both indoor and outdoor AR tracking with coordinated alerts could help return wandering Alzheimer’s patients safely to their homes.

So it seems that while entertainment is one possible use, AR is certainly not limited to that domain, but can help us stay healthy and improve outcomes when we need surgery. And what about that question about distraction while driving? Even that is being addressed: New tracking algorithms that help car systems estimate the head pose of a driver will enable systems to infer the driver’s focus of attention, triggering alerts and saving lives.

Create your own reality!

Brenda Wiederhold