A tool to assess nontechnical skills of perfusionists in the cardiac operating room

Roger D Dias; William Riley; Kenneth Shann; Donald S Likosky; David Fitzgerald; Steven Yule

doi:10.1016/j.jtcvs.2021.06.052

A tool to assess nontechnical skills of perfusionists in the cardiac operating room

J Thorac Cardiovasc Surg. 2023 Apr;165(4):1462-1469. doi: 10.1016/j.jtcvs.2021.06.052. Epub 2021 Jul 2.

Authors

Roger D Dias¹, William Riley², Kenneth Shann³, Donald S Likosky⁴, David Fitzgerald⁵, Steven Yule⁶

Affiliations

¹ STRATUS Center for Medical Simulation, Brigham and Women's Hospital, Boston, Mass; Department of Emergency Medicine, Harvard Medical School, Boston, Mass. Electronic address: rdias@bwh.harvard.edu.
² Cardiovascular Center, Tufts Medical Center, Boston, Mass.
³ Division of Cardiac Surgery, Massachusetts General Hospital, Boston, Mass.
⁴ Department of Cardiac Surgery, Michigan Medicine, University of Michigan, Ann Arbor, Mich.
⁵ College of Health Professions, Medical University of South Carolina, Charleston, SC.
⁶ Department of Surgery, Brigham and Women's Hospital/Harvard Medical School, Boston, Mass; Department of Clinical Surgery, University of Edinburgh, Edinburgh, Scotland.

Abstract

Objectives: This study aimed to develop the Perfusionists' Intraoperative Non-Technical Skills tool, specifically to the perfusionists' context, and test its inter-rater reliability.

Methods: An expert panel was convened to review existing surgical nontechnical skills taxonomies and develop the Perfusionists' Intraoperative Non-Technical Skills tool. During a workshop held at a national meeting, perfusionists completed the Perfusionists' Intraoperative Non-Technical Skills ratings after watching 4 videos displaying simulated cardiac operations. Two videos showed "good performance," and 2 videos showed "poor performance." Inter-rater reliability analysis was performed and intraclass correlation coefficient was reported.

Results: The final version of the Perfusionists' Intraoperative Non-Technical Skills taxonomy contains 4 behavioral categories (decision making, situation awareness, task management and leadership, teamwork and communication) with 4 behavioral elements each. Categories and elements are rated using an 8-point Likert scale ranging from 0.5 to 4.0. A total of 60 perfusionist raters were included and the comparison between rating distribution on "poor performance" and "good performance" videos yielded a statistically significant difference between groups, with a P value less than .001. A similar difference was found in all behavioral categories and elements. Reliability analysis showed moderate inter-rater reliability across overall ratings (intraclass correlation coefficient, 0.735; 95% confidence interval, 0.674-0.796; P < .001). Similar inter-rater reliability was found when raters were stratified by experience level.

Conclusions: The Perfusionists' Intraoperative Non-Technical Skills tool presented moderate inter-rater reliability among perfusionists with varied levels of experience. This tool can be used to train and assess perfusionists in relevant nontechnical skills, with the potential to enhance safety and improve surgical outcomes.

Keywords: assessment; behavioral markers; cardiac surgery; nontechnical skills; patient safety; perfusionist; teamwork.

Publication types

Research Support, N.I.H., Extramural

MeSH terms

Clinical Competence
Educational Measurement
Humans
Internship and Residency*
Operating Rooms*
Reproducibility of Results

Grants and funding

R01 HL146619/HL/NHLBI NIH HHS/United States