Interactive Augmented Reality Exposure Treatment

By Andreas Dünser

Augmented Reality (AR), a technology that overlays virtual content on the real world, has great potential for exposure treatment and in some cases may be more efficient and cost effective than Virtual Reality (VR). An environment that combines real and virtual elements seems to be especially suitable for such treatment because it enables displaying virtual fear stimuli in the real world.

Another benefit of using an AR system in exposure treatment is that it allows patients to use their own body to interact with the stimuli. However, this type of interactivity has so far not been fully implemented in AR-based exposure therapy systems. Still, studies have found that these systems are capable of inducing high levels of anxiety which is a necessary prerequisite for such a system to be effective. Considering that a virtual stimulus could not just move in the real word but also react and adapt to the client’s presence or actions (e.g., gestures) might make it even more effective for therapy.

We are developing an advanced interactive AR exposure therapy system that provides a high quality experimental test-bed and allows us to study the potential of using non-disruptive and interactive AR as a tool for exposure therapy. The novelty of our system lies in better integrating the real environment and the user into the system, using user actions/behavior as a control input to the system and creating a naturalistic environment. Our development is guided by the following aspects:

• Stable low cost hardware setup with off the shelf components
• Natural environment: Creation of a less disruptive and more “life-like” environment with using advanced tracking technology
• Environment recognition: Having virtual stimuli interact with the real environment and the client’s actions to further enhance interactivity and realism

Our first system integrated most of these aspects. This system allowed users to see and interact with virtual stimuli in a naturalistic environment. Using planar Natural Feature Tracking allowed us to display virtual stimuli on any sufficiently textured surface of the environment without the need for additional visual markers or tags. Interaction was supported through hand interaction above this surface, using a vision-based hand tracking algorithm and a basic gesture recognition technique.

However, the vision-based approach for tracking the user’s hand had some limitations. Therefore, we integrated the Microsoft Kinect into our AR system. The setup in Figure 1 shows a basic setup using the Kinect’s camera image for tracking. However, the system can also be setup using an additional Head Mounted Display with attached camera providing a more immersive direct view to the patient.

Using the depth information provided by the Kinect enables us to get better and more accurate hand tracking results. Furthermore, we get a more accurate model of the user’s environment. Using this information we can make the virtual stimuli “react” to the real environment and therefore create an even more realistic therapy environment. A virtual spider can now walk up, around, or behind real objects, and better react to user actions. For example (depending on the therapy stage and goals), a virtual fear stimulus can show approaching behavior to a patient’s hand, avoidance behavior, react to movements, be (partially) occluded by the patient, or even sit on the patient’s hands and follow their movements (see Figure 2). These elements create a more immersive and believable experience for the patient.

Future evaluation studies will focus on investigating the benefits of creating a more realistic AR-based exposure treatment environment and the effectiveness of such advanced interactivity.

Andreas Dünser, Ph.D.                                                                                                  The HIT Lab NZ University of Canterbury                                                                     New Zealand                                                                                               andreas.duenser@canterbury.ac.nz