Search
 
loader loading subjects...
cover
External Validation of a Machine Learning Based Delirium Prediction Software in Clinical Routine
Stefanie Jauk, Sai Pavan Kumar Veeranki, Diether Kramer, Stefan Högler, David Mühlecker, Erwin Eberhartl, Arthur Schueler, Christian Chvosta, Wolfgang Strasser, Reinhold Strasser, Werner Leodolter
93 - 100
10.3233/SHTI220353
Research Article
Studies in Health Technology and Informatics
Volume 293: dHealth 2022
Abstract

Background:

Various machine learning (ML) models have been developed for the prediction of clinical outcomes, but there is missing evidence on their performance in clinical routine and external validation.

Objectives:

Our aim was to deploy and prospectively evaluate an already developed delirium prediction software in clinical routine of an external hospital.

Methods:

We compared updated ML models of the software and models re-trained with the external hospital’s data. The best models were deployed in clinical routine for one month, and risk predictions for all admitted patients were compared to the risk ratings of a senior physician. After using the software, clinicians completed a questionnaire assessing technology acceptance.

Results:

Re-trained models achieved a high discriminative performance (AUROC > 0.92). Compared to clinical risk ratings, the software achieved a sensitivity of 100.0% and a specificity of 90.6%. Usefulness, ease of use and output quality were rated positively by the users.

Conclusion:

A ML based delirium prediction software achieved a high discriminative performance and high technology acceptance at an external hospital using re-trained ML models.

Download
Creative Commons License