Adult: Perioperative Management
A tool to assess nontechnical skills of perfusionists in the cardiac operating room

https://doi.org/10.1016/j.jtcvs.2021.06.052Get rights and content

Abstract

Objectives

This study aimed to develop the Perfusionists' Intraoperative Non-Technical Skills tool, specifically to the perfusionists' context, and test its inter-rater reliability.

Methods

An expert panel was convened to review existing surgical nontechnical skills taxonomies and develop the Perfusionists' Intraoperative Non-Technical Skills tool. During a workshop held at a national meeting, perfusionists completed the Perfusionists’ Intraoperative Non-Technical Skills ratings after watching 4 videos displaying simulated cardiac operations. Two videos showed “good performance,” and 2 videos showed “poor performance.” Inter-rater reliability analysis was performed and intraclass correlation coefficient was reported.

Results

The final version of the Perfusionists' Intraoperative Non-Technical Skills taxonomy contains 4 behavioral categories (decision making, situation awareness, task management and leadership, teamwork and communication) with 4 behavioral elements each. Categories and elements are rated using an 8-point Likert scale ranging from 0.5 to 4.0. A total of 60 perfusionist raters were included and the comparison between rating distribution on “poor performance” and “good performance” videos yielded a statistically significant difference between groups, with a P value less than .001. A similar difference was found in all behavioral categories and elements. Reliability analysis showed moderate inter-rater reliability across overall ratings (intraclass correlation coefficient, 0.735; 95% confidence interval, 0.674-0.796; P < .001). Similar inter-rater reliability was found when raters were stratified by experience level.

Conclusions

The Perfusionists' Intraoperative Non-Technical Skills tool presented moderate inter-rater reliability among perfusionists with varied levels of experience. This tool can be used to train and assess perfusionists in relevant nontechnical skills, with the potential to enhance safety and improve surgical outcomes.

Section snippets

Study Design and Setting

This study involved 2 sequential phases. In phase 1, an expert panel was assembled to (1) review the current literature on nontechnical skills in the operating room, (2) adapt a behavioral taxonomy to the perfusionist's context, and (3) generate an initial assessment tool for testing. Subject-matter experts met both in-person and remotely via videoconferencing. Phase 2 involved the observation and rating of simulated cardiac surgery videos by perfusionists during a dedicated 75-minute workshop

Results

A total of 100 participants completed ratings of at least 1 video. Participants who were not perfusionists (N = 10) and those who did not complete ratings of all 4 videos (N = 30) were excluded from the analysis. Our final sample included 60 perfusionists, who rated 4 cardiac surgery simulations using the PINTS tool. The experience in years of these raters is shown in Figure 2.

For each video, raters provided scores for 16 behavioral elements and 4 behavioral categories. Twenty ratings per video

Discussion

In this study, we have developed a nontechnical skills taxonomy and assessment tool specific for perfusionists. Our findings demonstrate that the PINTS tool presents a moderate inter-rater reliability, consistent across different levels of clinical experience and different domains of nontechnical skills. Additionally, the study results show that the PINTS tool is able to differentiate between poor and good nontechnical skills in simulated cardiac surgery videos, corroborating its construct

Conclusions

Perfusionists are essential members of the cardiac surgery team and their nontechnical skills play an imperative role in ensuring patient safety in the operating room. The PINTS taxonomy and rating instrument developed and examined in this study can be applied for training and assessment of perfusionists’ nontechnical skills for both research and quality improvement initiatives. As an assessment tool, the data gathered from applying this instrument to both novice and senior perfusionists,

References (35)

  • D.A. Cooley et al.

    The past 50 years of cardiovascular surgery

    Circulation

    (2000)
  • L. Melly et al.

    Fifty years of coronary artery bypass grafting

    J Thorac Dis

    (2018)
  • A.L. Axtell et al.

    Correlation of cardiopulmonary bypass duration with acute renal failure after cardiac surgery

    J Thorac Cardiovasc Surg

    (2019)
  • R.A. Baker et al.

    Report from AmSECT’s international consortium for evidence-based perfusion: American Society of Extracorporeal Technology Standards and Guidelines for Perfusion Practice:2013

    J Extra Corpor Technol

    (2013)
  • R.D. Dias et al.

    Dissecting cardiac surgery: a video-based recall protocol to elucidate team cognitive processes in the operating room

    Ann Surg

    (2019)
  • D. Wiegmann et al.

    A human factors analysis of cardiopulmonary bypass machines

    J Extra Corpor Technol

    (2009)
  • R.D. Dias et al.

    Development of an interactive dashboard to analyze cognitive workload of surgical teams during complex procedural care

    IEEE Int Interdiscip Conf Cogn Methods Situat Aware Decis Support

    (2018)

    • Cited by (5)

    This project was funded by the National Heart, Lung, and Blood Institute (R01HL146619).

    This study was approved by the Partners Institutional Review Board (Protocol #: 2020P000159, Approval Date: 02/06/2020), and a consent procedure was embedded onto an electronic survey.

    View full text
    © 2021 by The American Association for Thoracic Surgery