What specific knowledge, skills and attitudes does the endovascular expert operator possess? And, which stages do novice candidates have to pass through on their way to proficiency?

What specific knowledge, skills and attitudes does the endovascular expert operator possess? And, which stages do novice candidates have to pass through on their way to proficiency?

Recently, I defended the PhD-thesis: “Developing Endovascular Expertise – Aspects of Incorporating Virtual Reality Simulations”. The following is a summary of the encouraging results of assessment of human performance in a novel simulated setting.

The overall aim was to characterise the development of endovascular expertise and construct assessment instruments that capture this process.

“Endovascular (EV) procedures have replaced traditional open surgery in several vascular anatomical regions putting pressure on educating more operators. As for open surgery, the training method still follows the traditional apprenticeship model in which a supervising expert bases assessment of the trainee on subjective observation of performance. The feedback provided to the trainee on performance tends to be unstructured and this method has been criticized for lack of transparency with potential observer bias. Most patients treated endovascularly are awake, thus, the communication between the supervisor and the trainee during these procedures is restricted that limits feedback to the trainee. Available assessment instruments (rating scales and checklists) of performance have not been able to differentiate performances between novices and experts questioning validity. Likewise, the simulators’ built-in metrics for objective assessment of performance have not convincingly demonstrated good results of validity and reliability.

The fist part of the thesis was a comprehensive characterisation of the various facets of developing EV procedural expertise. The data were based on a review of the literature and an experimental study of seven EV operators’ performances on a simulator with subsequent interview applying the method of retrospective verbalisation. This investigation provided a novel global rating scale – SAVE (Structured Assessment of endoVascular Expertise) – that consists of 29 specific items. Technical skills were divided in two categories of internal- and external technique according to the operator’s focus of visual attention. In addition, the scale covers complex cognitive skills and reflects a much broader view on procedural EV expertise compared to prior rating scales.

The second part explored the construct validity and reliability of assessment with the SAVE scale in a simulated setting with novel virtual patient cases and a simulation lab with enhanced realism. The proctor took an active role of posing procedural questions. Twenty operators performed a simulated iliac artery stenting procedure. The results demonstrated good correlation between clinical experience and simulator performance with high inter-rater reliability. It was possible to discriminate performances between participants with experience levels up to 500 procedures, and only the experts (>5.000 procedures) obtained maximum scores on the scale. This study was the first not demonstrating any ceiling effect of assessment of performance with a rating scale in simulated EV procedures. The built-in simulator metrics for objective assessment of EV performance did not prove valid.”

The third part studied the association between testing fine-motor digital skills (aptitude) and EV performance. A novel aptitude test was developed for this purpose. The predictive value of testing novices’ aptitude to skills acquisition in a simulated task could not be proven. In the group of operators with EV experience the aptitude test result with the non-dominant hand correlated with their simulator performance. This supports the theory that innate fine-motor abilities probably are less important for skills acquisition among novices and the fine-motor skills pertained to EV expertise most likely develop during clinical practice in the angiosuite.

In conclusion, developing EV expertise is multi-facetted. In addition to basic knowledge these procedures comprise acquisition of procedural complex cognitive skills and unique technical skills upon handling EV tools. It is possible to obtain valid and reliable assessments of EV performance with the global rating scale SAVE without reaching a ceiling effect. The novel scale is suggested applied to the design of curricula and courses (testing for learning) as well as for credentialing purposes (testing of learning).

These encouraging results, of assessment of endovascular competence within a novel simulated setting, demonstrates the importance of physicians and engineers working closely together. It is now the time for the medical specialties to embrace this promising technology and stimulate further development and implement simulators into daily practices.

References:
Bech B, Lönn L, Schroeder TV, Räder SB, Ringsted C.
Capturing the Essence of Developing Endovascular Expertise for the Construction of a Global Assessment Instrument.
Eur J Vasc Endovasc Surg 2010;40(3):292-302.

Bech B, Lönn L, Falkenberg M, Bartholdy NJ, Räder SB, Schroeder TV, Ringsted C.
Construct Validity and Reliability of Structured Assessment of endoVascular Expertise in a Simulated Setting.
Eur J Vasc Endovasc Surg 2011;42(4):539-48.