Healthc Inform Res Search


Healthc Inform Res > Volume 28(3); 2022 > Article
Park and Cha: Application Strategies for Artificial Intelligence–based Clinical Decision Support System: From the Simulation to the Real-World
Clinical decision support systems (CDSS) have entered a new age thanks to algorithms based on artificial intelligence (AI) [1]. AI has shown the potential to enhance the precision and suitability of CDSS and attracted widespread interest from physicians, hence expanding its potential applications to nearly every aspect of contemporary medicine, from pharmacogenetics to public health [2,3].
In-hospital nursing care could be one of the most significant AI-CDSS use cases [4]. AI-CDSS can assist hospital nurses in providing high-quality care in an effective and efficient manner. Patient safety-related measures such as minimizing pressure injuries and falls are a top concern [5]. Recent efforts have demonstrated the potential benefits of AI-CDSS in various sectors (Figure 1).
However, the application of AI-CDSS in a real-world setting is challenging. Whereas the majority of the literature on AI development has been produced in simulation settings (i.e., on researchers’ well-controlled stand-alone servers and in silico studies), the application of AI models to the real world necessitates connecting to extremely complex and restricted environments involving numerous interactions, particularly with healthcare providers [6].
A five-phase process of evaluating an AI-CDSS for healthcare, from phase 0 to phase 4, has been proposed [7]. Phase 0 involves identifying users’ needs, ensuring data quality, and prototyping algorithms. Phase 1 is algorithm optimization with usability tests. Phase 2 involves measuring performance in practical real-world settings, along with measurements of clinical outcomes. Phase 3 is for identifying adverse events, and phase 4 involves post-deployment surveillance. This framework is persuasive, since its five-phase approach is widely used in drug and device evaluation process [8].
Two environments are associated with the five-phase process. First, in a simulation environment where an algorithm is produced, input variables are selected with few restrictions. If technically possible, derivative variables are liberally generated. The pool of variables is fixed and predetermined. In a real-world setting, it is emphasized that variables should be collected in economic ways, with interpretable transformations, and their time-dependent availability should be taken into consideration (Figure 2).
The second environment is the real-world setting, where it is essential to ensure that the accuracy of algorithms is maintained. The model should perform in a way that is clinically useful. In the real-world context, the output variables of algorithms have varying implications. An output variable should be selected based on its specific therapeutic value, with a focus on the difference that these prediction models could make by influencing users’ decisions and behaviors. A prediction is valuable only if it alters decisions and actions in the field.
The AI-CDSS application procedure is fundamentally dependent on multidisciplinary teams of devoted members, including experts in computer science, healthcare disciplines, and the social sciences. This team should lead the process, which consists of multiple iterations as opposed to a single intervention. It is expected that the probability of success will be increased by reliable and modern infrastructure, ongoing training, and robust communication. As mentioned above, a new era for AI-CDSS has been inaugurated, and now human intelligence will pave the way for the future.


Conflict of Interest

No potential conflict of interest relevant to this article was reported.

Figure 1
Screenshots of a nursing AI-CDSS for pressure injury. The system is designed to predict patients at risk of developing a pressure injury, and it also provides a personalized interpretation of the components that contributed to the outcome. AI-CDSS: artificial intelligence clinical decision support system.
Figure 2
The process of applying an AI-CDSS to a real-world setting. The procedure demonstrates the distinction between algorithms utilized in practice. In each configuration, variables and algorithms are seen and selected differently. AI-CDSS: artificial intelligence clinical decision support system.


1. Lisboa PJ, Taktak AF. The use of artificial neural networks in decision support in cancer: a systematic review. Neural Netw 2006 19(4):408-15.
crossref pmid
2. Fleming N. How artificial intelligence is changing drug discovery. Nature 2018 557(7707):S55-S57.
crossref pmid
3. Cvetkovic B, Janko V, Romero AE, Kafali O, Stathis K, Lustrek M. Activity recognition for diabetic patients using a smartphone. J Med Syst 2016 40(12):256.
crossref pmid
4. Clancy TR. Artificial intelligence and nursing: the future is now. J Nurs Adm 2020 50(3):125-7.
crossref pmid
5. Buchanan C, Howitt ML, Wilson R, Booth RG, Risling T, Bamford M. Predicted influences of artificial intelligence on the domains of nursing: scoping review. JMIR Nurs 2020 3(1):e23939.
crossref pmid pmc
6. Artificial intelligence in health care: within touching distance. Lancet 2017 390(10114):2739.
crossref pmid
7. Park Y, Jackson GP, Foreman MA, Gruen D, Hu J, Das AK. Evaluating artificial intelligence in medicine: phases of clinical research. JAMIA Open 2020 3(3):326-31.
crossref pmid pmc
8. Lipsky MS, Sharp LK. From idea to market: the drug approval process. J Am Board Fam Pract 2001;14(5):362-7.
crossref pmid
Share :
Facebook Twitter Linked In Google+ Line it
METRICS Graph View
  • 0 Crossref
  •     Scopus
  • 2,862 View
  • 289 Download
Related articles in Healthc Inform Res


Browse all articles >

Editorial Office
1618 Kyungheegung Achim Bldg 3, 34, Sajik-ro 8-gil, Jongno-gu, Seoul 03174, Korea
Tel: +82-2-733-7637, +82-2-734-7637    E-mail:                

Copyright © 2024 by Korean Society of Medical Informatics.

Developed in M2community

Close layer
prev next