OBJECTIVE: The term "active larynx" is a nonspecific and subjective term used by otolaryngologists to describe laryngeal inflammation that can influence the timing of airway reconstruction. We sought to measure the reliability of visual assessments of laryngeal inflammation for later scale development. STUDY DESIGN: A cross-sectional study. SETTING: Pediatric tertiary care center. METHODS: We created an image library from a direct laryngoscopy and bronchoscopy database. Blinded judges were asked to rate the characteristics of laryngeal inflammation (edema, erythema, cobblestoned appearance, and ventricular eversion
5-point Likert scale), the overall "activeness" of the larynx (10-point scale), and whether laryngeal inflammation would influence a delay in reconstructive surgery (yes/no). A tentative scale was also constructed. Intraclass correlations with 2-way random effects, and Fleiss's κ were used to evaluate interrater reliability. The convergent and discriminant validity of the tentative scale were measured. RESULTS: Three pediatric otolaryngologists reviewed 15 larynges for a total of 45 image ratings. Intraclass coefficients indicated substantial agreement for edema (0.76) and erythema (0.83) and moderate agreement for ventricular eversion (0.58). Cobblestoning had low agreement (intraclass correlation coefficient [ICC] <
0.20). The agreement was substantial for overall "activeness" (ICC 0.76) and moderate for whether inflammation would delay surgery (ICC 0.47). By Fleiss's κ, edema and erythema had moderate agreement (0.50 and 0.61, respectively), whereas all others had poor agreement. The convergent and discriminant validity of the tentative scale were reassuring. CONCLUSION: While the reliability of laryngeal inflammation by visual assessment is variable, the creation of an active larynx scale appears feasible.