Augmenting Mental Models

➣ By Egon L. van den Broek et al.

With recent technological developments, Virtual Reality (VR) has become more and more accessible for various application fields such as medicine, military, education, and product design. The possibility to interactively explore virtual worlds has become an important concept to support education, improve training of skills, or enhance human communication. It is generally acknowledged that the interactive characteristic and vivid imagery of VR improves the user’s subjective experience, not only increasing learning performance but also encouraging discussions with other users. However, despite all technological developments, creating VR remains a complex and expensive business. For many companies the practical implications of VR outweigh the practical benefits. Therefore, after decennia of VR research, the question remains:

What benefits does VR provide that is are not provided by other, less advanced technologies?

To understand the benefits of low-fidelity VR for its users (e.g., see Figure 1), we must first understand the effect VR has on human cognition. The main effect of VR on human cognitive processing is three folded, as VR can augment users’:

1. (multimodal) perceptual information processing by presenting information via, multimodal displays; e.g., large immersive visual displays that surround the users. Moreover, unlike other computermediated environments, VR provides the users with auditory, haptic, tactual, or even olfactory feedback, stimulating other perceptual processes as well.

2.motor processing by facilitating the users to perform natural interactions. Not only can users practice specific skills in VR, but also they can learn about the functionality of complex objects.

3.memory processing via increased perceptual and motor processing in VR, which facilitates the information storage in memory in a more accurate and complete manner than in other computer-mediated environments. Thus, VR aids the generation of ideas or mental models.

A mental model is a complex concept for which a broad range of definitions have been used. We define a mental model as the (partial) mental representation of the dynamics of the external world. As such, mental models aid understanding, provide explanations, and help predicting future situations; see Figure 1 and 4.

In many application fields such as medicine, VR can improve mental models. However, people have mental models of almost everything in their environment ranging from simple to complex items. Without aiming to present an exhaustive list, we present three characteristics that indicate the potentially added value of VR in generating mental models:

Team Work

When starting a project in which a new team is formed, the members might have different backgrounds, use their own jargon, and adopt their own mental models, see Figures. 2 and 3. Communication might be difficult and miscommunication and misunderstanding lie in wait. Having a shared mental model might prevent these problems as it contains the understanding and/or the commonality of individual mental models (at least by a part of the group). Shared mental models are created by shared experiences or shared familiarization and/or by communication and negotiation. VR fully supports these processes and is, therefore, considered an important means in problem-solving and decision-making processes.

Describing Abstract Concepts

VR is useful when abstract concepts are involved. It is easier to outline a mental model of something when different channels of information can be used; i.e., the principle of triangulation. Using solely words to explain an abstract concept might lead to different interpretations, as the interpretation of these words depends on individuals’ mental models. Using VR can help avoiding these different interpretations.

Complex Environments

As already was denoted and is long known in psychology, people benefit from multimodal information as opposed to unimodal information; see Figure 1. VR is per definition multimodal and, as such, is par excellence suitable in presenting information, in particular complex information. However, multimodal presentation of information has its downside. Humans are very sensitive to asynchrone presentation of modalities. A latency of even 50 msec. can cause a disturbance with the user, possibly without that the user is consciously aware of it.

Figure 4: Placing virtual products in a realistic context can further improve the users’ mental model.

In practice, these three characteristics are often intertwined. An example of an area where these characteristics often come together is risk management; i.e., the logical and systematic identification, analysis, treatment, and monitoring of the risks involved in an activity. In short, risk management is providing decision support. In general, crisis management involves i) teamwork as various stakeholders participate; e.g., police, firemen, and healthcare personal; ii) abstract concepts; e.g., people’s experience with events; and iii) complex environments; a crisis takes place under highly dynamic, often hectic circumstances. Therefore, (government) agencies often have real world practice drills to ensure all stakeholders are trained to handle such crisis. VR reality has already shown its use under such circumstances.

Although mental models are a well known concept, the operationalization of this concept is weak. Consequently, although always relevant, mental models are hardly ever applied explicitly in VR studies and remain an umbrella concept. Although touching upon the essence of VR, for the time being, the concept of mental models is bound to research and not applications. Par excellence, VR can be utilized to explore people’s mental models further as it facilitates multimodal interaction; see Figures 1 and 4. Exploration of concepts and thoughts as well as the communication of them with other people is all supported by VR technology. As such, VR is deemed to bring science a significant step further in understanding and applying mental models in practice.

Acknowledgments: The authors gratefully acknowledge the support of the Dutch Innovation Oriented Research Program, Integrated Product Creation and Realization (IOP-IPCR), of the Dutch Ministry of Economic Affairs. Additionally, we thank TXchange (URL: http://www.txchange.nl/) for granting us the permission to use their graphics (i.e., Figures 1, 2, and 4) as illustration in this article.

Egon L. van den Broek, Ph.D.
Frank Meijer, M.A.
University of Twente

Renske B. Landman, M.Sc.
ErgoS Engineering and Ergonomics
The Netherlands

vandenbro ek@acm.org
frankmeij@gmail.com
r.b.landman@alumnus.utwente.nl