By Alessandro Tognetti et al.
People express their mental states through gesture, posture, facial expression, eye gaze, prosody and vocal tones. Hidden physiological parameters such as Heart Rate (HR), Heart Rate Variability (HRV), Respiration Rate (RR), and electrodermal response (EDR) are somehow correlated to emotions and thoughts. Ubiquitous monitoring of physiological and behavioral correlates of complex mental states might prove instrumental in providing insight in dealing with mental disorders.
Our group is involved in the development of fully wearable interfaces for unobtrusive monitoring of physiological signs, activity and movement, eye gaze and facial expression to explore their potentialities in the management of mental disorders.
Our bio-monitoring research is mainly focused on evaluating human Psycho-Physiological State (PPS) by processing peripheral and central signals acquired by non-invasive monitoring systems (Fig. 1). The aim is to identify differences between healthy and pathological subjects as well as healthy (pathological) subjects in different psycho-physiological states. Several domains are covered ranging from biomedical signal processing, sensors, embedded electronics, advanced analysis and computational intelligence. Most of the physiological signals are acquired by means of textile- based systems, Electrocardiogram (ECG) and breathing activity through sensing T-shirts and EDR through sensing gloves. Concerning Central Nervous System (CNS), standard instrumentation such as clinical and high resolution Electroencephalogram (EEG) are used as a gold standard for cognitive investigation. Moreover, a compact head mounted eye-gaze tracker has been developed in order to investigate the functional connectivity between pupil size variation or gaze-point set and PPS.
Wearable interfaces for full body monitoring (Fig. 2) have been developed through the textile integration of elastic sensors based on conductive elastomer (CE) materials. CE materials show piezo-resistive properties – a fabric deformation can be related to a sensor electrical resistance variation. CE materials can be integrated into elastic fabrics without changing the substrate mechanical properties, thus maintaining user comfort during the monitoring phase. These innovative sensing garments allow the monitoring of body posture and gesture in rehabilitation and human machine interaction for disabled people. Furthermore, these systems were successfully employed in behavioral studies with functional MRI during specific motor tasks. The main advantages of the described technologies are the overall system lightness, flexibility and unobtrusiveness, and the low costs of the textile process. Current research is oriented towards the combined use of inertial and textile sensors in a data fusion approach in order to address the limitations of the systems that are currently available.
A new paradigm for the treatment of Autistic Spectrum Disorders (ASD) has been conceived and implemented in our lab by means of the FACE android which is used as an interactive, physical display of emotional facial expressions; a room in which the human-android interaction takes place is equipped with motorized cameras, directional microphones, smart wearable sensors for physiological signals acquisition, and other acquisition and user interaction systems as shown in Fig. 3. The physiological and gestural wearable interfaces are being used in several research projects focused on basic investigation, assessment, and treatment of psychological related aspects.
In the EU-funded PSYCHE project, the described wearable bio-monitoring systems are used for the study and treatment of bipolar patients.
Within the EU-funded INTERSTRESS project, whose aim is the physiological stress treatment through virtual reality experiences and continuous patient monitoring, a mobile personal bio-monitoring system is under development. This system will unobtrusively perform a continuous tracking of patients’ physiological (HR,HRV, RR) and behavioral status. Behavioral assessment is not intended for stress identification but for the definition of physiological measurements context.
Recent studies within the EU-funded CEEDS project, are oriented towards the capture and classification of explicit and implicit navigation through mental states to investigate the process of discovering hidden data in huge multi-dimensional datasets.
Alessandro Tognetti, Ph.D. Enzo Pasquale Scilingo, Assistant Professor, Ph.D. Gaetano Anania, M.Eng Nicola Carbonaro, Ph.D. Antonio Lanatà, Ph.D. Federico Lorussi, Ph.D. Daniele Mazzei, Ph.D. Gaetano Valenza, M.Eng Danilo De Rossi, Full Professor University of Pisa Italy a.tognetti@centropiaggio.unipi.it
About Brenda Wiederhold
President of Virtual Reality Medical Institute (VRMI) in Brussels, Belgium.
Executive VP Virtual Reality Medical Center (VRMC), based in San Diego and Los Angeles, California.
CEO of Interactive Media Institute a 501c3 non-profit
Clinical Instructor in Department of Psychiatry at UCSD
Founder of CyberPsychology, CyberTherapy, & Social Networking Conference
Visiting Professor at Catholic University Milan.
