A Virtual Training System for Children with Upper Extremity Disability: Providing Real-time Interactive Feedback for Hand Rehabilitation

➣ By Kup-Sze Choi

Hand dexterity is essential for many activities of daily living (ADL), e.g., buttoning or tying shoelaces. Among them, writing and drawing are of particular importance to children who may spend up to 60% of their school time on handwriting. Proper handwriting requires well-coordinated fine motor movements of the fingers. For children suffering from motor impairment, resulting from cerebral palsy, for example, it is difficult to control the small muscles of various fingers in order to appropriately adjust the pencil’s position in their hand. Rubber pencil grips, pen tablets and other assistive gadgets have been used to help these children regain their handwriting ability. However, these devices do not provide active feedback in the training process, and a tutor is required to assess the children’s performance, to demonstrate ways of writing, or to provide guidance by holding and maneuvering their hands. By taking advantage of Virtual Reality (VR) technology, the rehabilitation process can be performed in a more interactive and efficient manner.

VR-based Training System

A VR-based system has been developed to facilitate hand rehabilitation through computerized training. The objectives are to enable self-practice by providing interactive visual and haptic cues as guidance, and by monitoring user performance automatically with quantifiable performance metrics. The hardware consists of a generic personal computer and the Phantom Omni haptic device made by SensAble Technologies, Inc. The software of the system is built using C/C++, OpenGL and OpenHaptics. The haptic device has a pen-like stylus, which can be grasped and maneuvered in a 3-D space. Using a piece of virtual paper laid horizontally, users practice handwriting with the stylus as if they were writing with a real pencil (see Figure 1). When the tip of the virtual pencil-tip contacts the paper, the strokes are displayed simultaneously on the screen, while feedback forces, i.e., contact forces and haptic cues, are generated to drive the haptic device.

Visual and Haptic Cues

Training begins by showing the image of the complete character or pattern in the background. The user is required to draw, stroke by stroke, by following the visual cues provided. The visual cues are displayed in the form of guidelines superimposed on top of the background image, as shown by the yellow lines in Figure 2. Users are guided to draw the strokes one by one sequentially.

The users are also guided by interactive forces when drawing along the guidelines. If the virtual pencil-tip deviates, leaving the writing surface or moving laterally away from a guideline, forces are generated to pull it back to the guideline. The forces are computed in real time in response to the user’s movements. In the system, two kinds of forces are provided to: (i) move the pencil- tip from the starting point of a guideline towards its end, or (ii) to pull the pencil- tip back to the guideline if it is being moved away.

Quantitative Performance Assessment

User performance can be analyzed in detail using the quantitative metrics provided by the system. Timing data including the time taken to complete a drawing (completion time), the time taken to draw individual strokes (stroke-drawing time), and the time elapsed during transitions between consecutive strokes (transition time) are automatically recorded. Furthermore, undesirable maneuvers, such as inair stroke drawing and on-paper transitions, are measured. Path length, deviation from the ideal path, forces applied by the user, and the trajectory of the virtual pencil tip (shown in Figure 3) can also be recorded by the system.

Evaluation and Future Work

The system has been implemented in a pilot study for children with cerebral palsy to practice writing Chinese characters, as shown in Figure 1. In a two-week study where practice was conducted two times a week during 20-minute sessions, the subjects showed improvements in handwriting speed and accuracy (average completion time and path length reduced by 50% and 30% respectively), especially for those diagnosed with a mild degree of deficiency. The legibility of real handwriting on paper was also improved, as evaluated by language teachers. Featuring interactive feedback and quantitative assessment, the system has the potential to facilitate selfpractice with minimal supervision from therapists. In the next version, tablet devices will be adopted to offer a natural user interface where the drawings are displayed directly under the stylus end-point. Clinical trials with a larger sample size and longer study period will be conducted to further evaluate the effectiveness of the system.

The author would like to acknowledge the support of the Hong Kong Red Cross Princess Alexandra School and its OT team in the project.

Kup-Sze Choi, Ph.D.
School of Nursing
The Hong Kong Polytechnic University
Hong Kong
kschoi@ieee.org