INTRODUCTION:

technical skills acquisition is considered to be of paramount importance in surgical training. Yet, formal assessment of technical skills is the weakest and less developed area. Currently available resources to evaluate technical skills are largely subjective, and lack of validity and reliability. Direct observation, one of the most frequently used methods, is largely biased by interpersonal subjectivity and personality traits. We propose the creation and use of a new procedure-specific tool for objective assessment of technical skills in surgery to evaluate validity and reliability.

MATERIALS AND METHODS:

laparoscopic cholecystectomy and Lichstenstein’s inguinal hernia repair were the chosen procedures.

Three groups of comparison were defined according to surgical expertise:

initial, intermediate, and experts. Surgeries were videorecorded in real time without identification of the patient or the surgeon. Tapes without any posterior edition were assigned to two expert surgeons in a blind and randomized sequence. A newly proposed procedure-specific rating scale was used for evaluation, as well as Reznick’s OSATS global scale. Kruskal-Wallis non-parametric test was used to assess validity. p 0.8 granted reliability.

RESULTS:

from April 2010 to December 2012 36 laparoscopic cholecystectomies and 31 inguinal hernia repairs were recorded. Significant difference was found among groups of comparison for every item (p<0.05). Crhonbach’s ? was largely over 0,80 for both procedures. There was no significant difference between the assessments of the two raters. There was no significant difference between our newly proposed procedure-specific instrument and Reznick’s OSTATS.

DISCUSSION:

the objective assessment of technical skills in surgery is feasible and useful. The tool we proposed showed construct validity and reliability. Video recording of surgical procedures grants durability over time to an ephemeral phenomenon. The objectivity is based on the explicit statements and quantification of every step to be evaluated, and the blind randomization and anonymous treatment of the sample. Sharing the same quality criteria between evaluators is of paramount importance to reach satisfactory results. The process of evaluation always implies a shortened view of the reality.