Evaluation of Electronic Medical Record Artificial Intelligence Screening Tools for Undiagnosed OSA
Sam Rusk, BS, Yoav N. Nygate, MSc, Fred Turkington, BS, Chris R. Fernandez, MS, Nick Glattard, MS, Nathaniel F. Watson, MD, MSc
The STOP-Bang is a concise, simple and widely adopted obstructive sleep apnea (OSA) screening tool. However, it has limited predictive ability and is susceptible to subjective reporting bias.
Artificial Intelligence (AI) methodologies can be utilized together with existing data in electronic medical records (EMRs) to create new screening tools to increase diagnostic sensitivity and facilitate discovery of preclinical OSA phenotypes.
Stratified random sampling with proportionate allocation was applied to a database of N>10,000 retrospective diagnostic clinical polysomnography (PSG), selected by evidence grading standards, with controls applied for OSA severity, diagnoses; sleep, psychiatric, neurologic, neurodevelopmental, cardiac, pulmonary, metabolic disorders, medications; benzodiazepines, antidepressants, stimulants, opiates, sleep aids, demographic groups of interest; sex, adult age, pediatric age, BMI, weight, height, and patient-reported sleepiness, to establish representative N=100 Adult and N=100 Pediatric samples.
Double Blinded scoring was prospectively collected for each sample by 3 experienced RPSGT certified sleep technologists randomized from a pool of 9 scorers.
Sensitivity (PA), Specificity (NA), Accuracy (OA), Kappa (K), and 95% Bootstrap CI’s are presented for sleep stages, OSA/CSA, hypopnea 3%/4%, arousals, limb movements, Cheyenne-Stokes respiration, periodic breathing, atrial fibrillation, and other events, and normative, mild, moderate, and severe OSA categories for global-AHI and REM-AHI.
Results for Sleep Staging and OSA Severity Diagnostic Accuracy are summarized.
CCDS-OSA ROC-AUC exceeded STOP-Bang and P-Bang for both sleep study collections, resulting in AHI>15 ROC-AUC values of 0.73 and 0.71 (CCDS-OSA) compared to AHI>15 ROC-AUC values of 0.68 and 0.69 (STOP-Bang).
Additionally, we analyzed the Gini feature importance ranking of the trained CCDS-OSA model to evaluate which variables showed highest predictive value of OSA.
The ranking revealed the top 5 features were the five physiologic based STOP-Bang parameters, followed by EMR based physiologic measurements such as HDL, triglycerides, systolic BP, and disease conditions such as diabetes, hypertension, and depression.
This study shows that while STOP-Bang contains data critical to OSA screening, a variety of other EMR-based parameters can improve performance of OSA detection.
AI-based EMR screening can provide a critical tool for more systematic and accurate screening of undiagnosed sleep apnea.
Nationwide standards facilitating patient EMR data interoperable health information exchange, particularly the United States Core Data for Interoperability (USCDI CCDS), holds promise to foster broad clinical and research opportunities.
Resulting data sharing will allow application of AI screening tools at the population health scale with ubiquitous, existing EMR data to improve population sleep health.