Pilot Project Validates Tool to Assess Thyroidectomy Skills

Clinical Question: How valid, reliable and feasible is a tool developed at Johns Hopkins to measure the development of trainees’ operating room skills for thyroid surgery?

Background: The 2001 Accreditation Council for Graduate Medical Education (ACGME) Outcome Project calls for residency programs to objectively measure trainees for six core competencies: patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism and systems-based practice. Measuring trainees’ operative competence in surgical specialties may pose a challenge owing to the lack of standardized objective assessment tools.

Study design: Prospective validation study.

Setting: Department of Otolaryngology-Head and Neck Surgery, Johns Hopkins University School of Medicine; Biostatistics Center, Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore.

Synopsis: Johns Hopkins developed, implemented and pilot-tested a two-component (a test-based checklist and a global rating scale) assessment tool for thyroidectomy, a core procedure in otolaryngology-head and neck surgery. During a period of one year, a total of 94 evaluations were completed by a single faculty member for 17 trainees as they performed thyroid surgery. The median time to complete the evaluation was two minutes, taking between five and seven minutes at the beginning of the study and one minute at the end of the study. The study authors said the tool provided a structure for faculty to provide feedback and identify residents in need of remediation. The researchers found the major score difference for both elements of the tool was between intermediate and senior residents, which corresponds to the rapid improvement phase of a learning curve.

Bottom Line: The tool developed by Johns Hopkins implemented a feasible, valid and reliable evaluation instrument for the assessment of technical skills in thyroid surgery.

Reference: Diaz Voss Varela DA, Malik MU, Thompson CB, et al. Comprehensive assessment of thyroidectomy skills development: a pilot project. Laryngoscope. 2012;122(1):103-109.