Artificial Intelligence to Predict Billing Code Levels of Emergency Department Encounters
Authors: Jacob Morey, Richard Winters, Derick Jones
Journal: Annals of Emergency Medicine, September 2024
Conclusion: The AI model demonstrated high accuracy in predicting billing code levels for ED encounters, with significant potential for automating the coding process. By leveraging clinical notes, characteristics, and orders, the model could help reduce administrative burdens, save time, and minimize costs in the coding of ED encounters.
Methods:
- Data Collection: ED encounter data from January to September 2023 was collected from a health system. This included clinical notes, clinical characteristics, and orders.
- Model Development: An ensemble AI model, incorporating natural language processing (NLP) and machine learning techniques, was used to predict billing code levels. Explainable AI methods, such as Shapley Additive Explanations (SHAP), were utilized to identify the most important features contributing to the model’s predictions.
- Endpoints: The primary endpoint was predicting E/M professional billing codes (levels 2 to 5 and critical care). Secondary endpoints included performance at various decision boundary thresholds and assessing the generalizability of the model across different EDs.
Results:
- Sample Size: The study analyzed 321,893 adult ED encounters, with the following distribution of billing levels: level 2 (<1%), level 3 (5%), level 4 (38%), level 5 (51%), and critical care (5%).
- Model Performance:
- For level 4 and level 5 encounters, the model achieved:
- Area Under the Receiver Operating Characteristic Curve (AUC): 0.94 for level 4 and 0.95 for level 5.
- Accuracy: 0.80 for level 4 and 0.92 for level 5.
- F1-scores: 0.79 for level 4 and 0.91 for level 5.
- At a 95% decision boundary threshold for level 5 encounters, the model exhibited a precision/positive predictive value (PPV) of 0.99 and a recall/sensitivity of 0.57.
- Key Features: The most influential factors identified by SHAP values included critical care notes, the number of orders, discharge disposition, cardiology involvement, and psychiatry consultations.