Convolutional Neural Network Models for Automatic Pre-Operative Severity Assessment in Unilateral Cleft Lip
Meghan McCullough, MD, MS1, Steven Ly, PhD1, Alex Campbell, MD, MS2, Caroline Yao, MD, MS3, Mark Urata, MD, DDS1, Stefan Scherer, PhD1, William P. Magee, III, MD, DDS1.
1University of Southern California, Los Angeles, CA, USA, 2Operation Smile, Virginia Beach, VA, USA, 3Harvard University, Cambridge, MA, USA.
PURPOSE: Despite the wide range of cleft lip morphology, consistent scales to categorize pre-operative severity do not exist. Machine learning has been used to increase accuracy and efficiency in detection and rating of multiple conditions, yet it has not been applied to cleft disease. We test a machine learning approach to automatically detect and measure facial landmarks and assign severity grades using pre-operative photographs.
METHODS: Pre-operative images were collected from 800 unilateral cleft lip patients, manually annotated for cleft-specific landmarks and rated using a previously validated severity scale by eight expert reviewers. Five convolutional neural network (CNN) models were trained for landmark detection and severity grade assignment. Mean squared error (MSE) loss and Pearson correlation coefficient for cleft-width-ratio (CWR), nostril-width-ratio (NWR) and severity grade assignment were calculated.
RESULTS: All five CNN models performed well in landmark detection and severity grade assignment with the largest and most complex model, ResNet, performing best (MSE = 24.41, CWR correlation = 0.943, NWR correlation = 0.879, severity correlation = 0.892). The mobile-device compatible network, MobileNet also showed a high degree of accuracy (MSE = 36.66, CWR correlation = 0.901, NWR correlation = 0.705, severity correlation = 0.860).
CONCLUSION: Machine learning models demonstrate the ability to accurately measure facial features and assign severity grades according to validated scales. Such models hold promise for the creation of a simple, automated approach to classifying cleft lip morphology. Further potential exists for a mobile-phone based application to provide real-time feedback to improve clinical decision making and patient counseling.
Back to 2019 Abstracts