Multi-Institutional Validation of the Interarytenoid Assessment Protocol for Pediatric Laryngeal Cleft

Each study participant underwent microlaryngoscopy and bronchoscopy, including each step of the protocol below (Fig. 1):

Step 1: Suspension laryngoscopy to obtain a Cormack-Lehane grade I exposure, where possible.

Step 2: Insertion of laryngeal distending forceps to allow adequate visualization of the IAMH, without overdistension to avoid distortion of the anatomy. Appropriate distension positions the vocal folds at a 30-degree angle.

Step 3: Palpation of the interarytenoid area using a right-angle microlaryngeal probe.

Step 4: Objective assessment of IAMH. With the right hand braced on the patient’s face or the laryngoscope to prevent movement in or out of the mouth, the right angle laryngeal probe is swung anterolaterally from a resting place in contact with, but not distorting, the interarytenoid mucosa to an anterolateral point on the lateral glottis, namely: (1) above false vocal folds (FVFs), (2) at FVFs, (3) in ventricle, (4) at true vocal folds (TVF)s, and (5) below TVFs (Fig. 2).

Video Review

Reliability of the IAAP was determined by review of 30 endoscopic videos of the IAAP from the Seattle Children’s Hospital Otolaryngology Database, based on an overall sample size deemed appropriate for reliability testing based on a priori statistical consultation. Inclusion criteria for videos required sufficient quality to visualize the procedure. A total of 115 records were reviewed for inclusion, with 32 videos excluded due to inadequate visualization of the steps of the IAAP, eight excluded due to poor quality, and 45 excluded because we had already filled the target number for the IAMH category.

Ten participants from 10 different institutions each evaluated 30 de-identified endoscopic videos. All of the videos were recorded at one institution and rated by participants from external institutions. Raters were blinded to identifying information and the number of videos selected for each IAMH. Raters were provided with the IAAP Scoring Sheet with examples of the appearance of the right-angle probe at different IAMHs. Raters were instructed to only evaluate the anatomy based on the contact point of the microlaryngeal probe demonstrated by the surgeon, without attempting to make independent assessments of the surgical technique. Videos were edited to only show the IAAPs being performed, without additional procedures. The order of the videos reviewed was randomized on both occasions, and raters were blinded to their rating of the first video when reviewing the same video for the second time.

Each video was scored at two separate time points, three months apart, for a total of 500 separate reviews. Intra-class correlation (ICC) coefficients with two-way models were computed to evaluate inter- and intra-rater reliability. The degree of agreement was defined by the following categories: 0–0.20 slight, 0.21–0.4 fair, 0.41–0.60 moderate, 0.61–0.80 strong, and 0.81–1.0 near complete agreement. The R Project for Statistical Computing was used for analysis with significance of P<0.05.

RESULTS

Thirty endoscopic videos were collected for patients with a median (interquartile) age of 4.9 years (59 months; range: one month to 20 years). Thirty percent of patients were female (seven years; range, 3.2 months to 20 years) and 70% were male (four years; range: 1.1 months to 13.2 years). Ten fellowship-trained pediatric otolaryngologists from 10 different institutions participated in the IAAP reliability assessment. Both the first and second video assessments, which were two months apart, showed strong inter-rater reliability. On the first video assessment (Part 1), inter-rater reliability was 0.74 (95% confidence interval [CI] 0.63 to 0.84), and on the second video assessment (Part 2), inter-rater reliability was 0.75 (95% CI 0.63 to 0.85). Overall intra-rater reliability ranged from 0.49 to 0.89 with an overall test–retest reliability of 0.75 (95% CI 0.69 to 0.79), indicating strong agreement.

There were seven (23%) instances of 100% agreement on the same IAAP Classification among the 10 participants. Of these, three were “above FVF,” one was “at FVF,” two were “in ventricle,” and one was “at TVF.” In almost half, 14 (46.6%) raters chose IAAP classification levels within one level of each other. And in nine cases (30%), there were two or more levels of spread of IAAP classification amongst the 10 raters.

CONCLUSIONS

The interarytenoid protocol (IAAP) demonstrates strong inter- and intra-rater reliability in the assessment of IAMH evaluated through pictorial analysis. Given that the IAAP’s effectiveness is also dependent on the precision of surgical technique, however, the reliability observed in this study should be viewed as a reflection of the consistency of visual interpretation alone. Further research is needed to assess how variations in surgical technique may impact the overall reliability of the IAAP, ensuring a more comprehensive understanding of its clinical utility. Standardization of anatomical evaluations may improve our ability to perform more reliable outcomes studies of pediatric pharyngeal dysphagia in the future.