Plastic Surgery Research Council

Back to 2019 Abstracts


Hearing Hypernasality: A Novel Assessment of Cleft Speech Using Online Crowdsourcing
Anne M. Sescleifer, BS, Janna Webber, CCC-SLP, Caitlin A. Francoisse, MD, Jeffrey D. Rector, PhD, Alexander Y. Lin, MD, FACS.
Saint Louis University School of Medicine, St. Louis, MO, USA.

PURPOSE: Speech intelligibility is fundamental to social interactions, which is why speech assessments are such an important focus in the pre- and post-operative care of patients born with cleft palate. Online crowdsourcing of perceptual speech outcomes is a burgeoning technology, and this project represents the first time cleft speech has been evaluated by lay listeners in an online setting. We hypothesized that lay ratings of cleft speech would demonstrate high concordance with the ratings of speech experts.
METHODS: Six audio-phrases from videonasoendoscopies (VNEs) of children with cleft palate were selected for varying degrees of hypernasality: Katie-Likes-Cookies (KC), Tell-Ted-to-try (TT), Should-I-wash-the-dishes (WD), Peter-has-a-puppy (PP), Sissy-sissy-sissy (SS), and Zippers-are-easy-to-close (ZC). These recordings had previously been assessed by the cleft team's speech language pathologists (SLP) in clinic, and given a speech score based on the Pittsburgh Weighted Speech Score (PWSS). Phrases were presented to Amazon Mechanical Turk crowdsourced raters to rate for hypernasality on a Likert scale from 1 to 5, corresponding to the hypernasal component (0-4) of the PWSS. The survey page had clickable "gold standard" samples representing hypernasality none (hypernasal component 0 of PWSS), mild (PWSS hypernasal 2), and severe (PWSS hypernasal 4) for the rater to use as reference.
RESULTS: Audio was extracted from VNEs of 3 children with cleft palate, ages 4-8, with timing of recordings ranging from pre-surgical repair to 6.5 years followup. Speech clips were distributed in 68 surveys, resulting in 1,088 unique layperson ratings. When all speech ratings were averaged, Patient 1 crowd mean was 2.62 (SLP rated 2-3); Patient 2 crowd mean 1.76 (SLP rated 2); and Patient 3 crowd mean 2.66 (SLP rated 3). When rounded to nearest whole number for consistency with PWSS scale, all patients matched SLP ratings. To determine the accuracy of each specific phrase, residuals were calculated (layperson minus SLP, see Table), showing phrase accuracy ordered WD>PP>ZC>TT>SS>KC; Figure shows the Tukey posthoc pairwise comparisons statistical significance levels.
CONCLUSION: Online crowdsourced ratings of cleft speech for hypernasality are highly consistent with SLP ratings, and predicted SLP ratings in all 3 patients. Phrases had different layperson accuracies: WD~PP~ZC>TT>SS~KC. This novel technology has immediate translation in clinical speech assessments, especially for centers without SLPs or requiring further clinical corroboration.

MTurk Speech Results by Phrase
ChildAge, SexPost-op Follow-upSLP ScoreKC mean (SD)TT mean (SD)WD mean (SD)PP mean (SD)SS mean (SD)ZC mean (SD)
P14 yrs, F2 years2-32.25 (0.92)2.97 (0.99)2.93 (0.94)2.19 (1.03)2.75 (1.23)---
P28 yrs, M6.5 years21.56 (1.28)---1.49 (1.09)1.38 (1.18)1.40 (1.10)2.97 (1.08)
P35 yrs, MPre-op32.15 (0.90)3.32 (0.88)3.24 (0.95)3.49 (0.80)1.96 (1.16)1.78 (1.21)



Back to 2019 Abstracts