J Am Med Inform Assoc. 2017 May 1;24(3):472-480. doi: 10.1093/jamia/ocw136.

Predicting inpatient clinical order patterns with probabilistic topic models vs conventional order sets.

Chen JH¹, Goldstein MK^2,³, Asch SM^1,⁴, Mackey L⁵, Altman RB^1,^6,⁷.

Author information

1: Department of Medicine, Stanford University, Stanford, CA, USA.
2: Geriatrics Research Education and Clinical Center, Veteran Affairs Palo Alto Health Care System, Palo Alto, CA, USA.
3: Primary Care and Outcomes Research (PCOR), Stanford University, Stanford, CA, USA.
4: Center for Innovation to Implementation (Ci2i), Veteran Affairs Palo Alto Health Care System, Palo Alto, CA, USA.
5: Department of Statistics, Stanford University, Stanford, CA, USA.
6: Department of Bioengineering, Stanford University, Stanford, CA, USA.
7: Department of Genetics, Stanford University, Stanford, CA, USA.

Abstract

Objective:

Build probabilistic topic model representations of hospital admissions processes and compare the ability of such models to predict clinical order patterns as compared to preconstructed order sets.

Materials and Methods:

The authors evaluated the first 24 hours of structured electronic health record data for > 10 K inpatients. Drawing an analogy between structured items (e.g., clinical orders) to words in a text document, the authors performed latent Dirichlet allocation probabilistic topic modeling. These topic models use initial clinical information to predict clinical orders for a separate validation set of > 4 K patients. The authors evaluated these topic model-based predictions vs existing human-authored order sets by area under the receiver operating characteristic curve, precision, and recall for subsequent clinical orders.

Results:

Existing order sets predict clinical orders used within 24 hours with area under the receiver operating characteristic curve 0.81, precision 16%, and recall 35%. This can be improved to 0.90, 24%, and 47% ( P < 10 -20 ) by using probabilistic topic models to summarize clinical data into up to 32 topics. Many of these latent topics yield natural clinical interpretations (e.g., "critical care," "pneumonia," "neurologic evaluation").

Discussion:

Existing order sets tend to provide nonspecific, process-oriented aid, with usability limitations impairing more precise, patient-focused support. Algorithmic summarization has the potential to breach this usability barrier by automatically inferring patient context, but with potential tradeoffs in interpretability.

Conclusion:

Probabilistic topic modeling provides an automated approach to detect thematic trends in patient care and generate decision support content. A potential use case finds related clinical orders for decision support.