Higher Order Performance Assessments Using Error Enabled Simulations: A New Approach

➣ By Carla M. Pugh & Janis Cannon-Bowers

In the operating room environment, mastery of surgical skills requires both technical excellence and excellence in operative decision making. Recognizing and correcting errors for both of these skills is a vital component of surgical training. When the goal of training is achievement of expertise, deliberate practice should be strategically incorporated as part of the learning process. Many characteristics once believed to reflect innate talent are now believed to be the result of intense practice extended over many years. Proper implementation of deliberate practice requires strategically designed performance assessments and high quality, structured feedback. Performance-based assessments have been defined as behavior-related measurements in settings designed to emulate real-life contexts or conditions in which specific knowledge or skills are actually applied. In essence, performancebased assessments require evaluation during the performance of tasks that are valued in their own right. In contrast, pencil-and-paper tests derive their value primarily as indicators or correlates of other valued performances.

Overall, performance-based assessments are well positioned to measure complex skills including communication and disciplinary knowledge which are considered important competencies in medical and surgical training. Surgical procedure simulators that allow for error prevention, error recognition and error rescue offer training opportunities that may exceed those possible in the live setting as error avoidance and patient safety trump trainee independence. Figures 1-3 show images of ventral hernia repairs completed by three different surgical residents during a p e r f o rma n c e – based assessment grounded in an error framework. All of the participants were senior level residents (fourth or fifth year) from different training programs. In Figure 1, a number of errors can be seen: a) there are no anchoring stitches, b) the mesh does not completely cover the hernia defect, and c) the mesh is not completely flat or flush to the abdominal wall. In Figure 2: a) the anchoring stitches are not well positioned for the mesh size and b) the mesh does not completely cover the hernia defect. In Figure 3: a) the mesh is too large for the defect, b) the mesh is off-centered, and c) some of the mesh tacks are too far away from the mesh edge. This surgical procedure simulator allows for independent operative decision making which allows for error prevention, recognition and rescue. Figures 4-5 show the complete system.

Error-enabled simulations allow for higher order training and assessment. Moreover, in the process of recognizing and correcting errors and near misses, the possibility exists that the newly developed, behavioral adjustment may still result in an error or worsening of an already recognized error. The theoretical basis for this was described by Wirstad. In essence, as knowledge-based behaviors are employed when a situation is novel or unexpected, the resulting behavior represents a more advanced level of reasoning. Moreover, in these situations, the cognitive workload is typically greater than that noted for basic skill- or rule-based behaviors and may increase the likelihood of making errors. Recognizing and correcting errors is a vital component in the process of achieving mastery both for individuals and high functioning specialized teams.

Carla M. Pugh, M.D., Ph.D.
Janis Cannon-Bowers, Ph.D.
Center for Advanced Surgical Education, Northwestern University
Chicago, Illinois
cpugh@nmh.org