In-hospital nursing care could be one of the most significant AI-CDSS use cases [4]. AI-CDSS can assist hospital nurses in providing high-quality care in an effective and efficient manner. Patient safety-related measures such as minimizing pressure injuries and falls are a top concern [5]. Recent efforts have demonstrated the potential benefits of AI-CDSS in various sectors (Figure 1).

However, the application of AI-CDSS in a real-world setting is challenging. Whereas the majority of the literature on AI development has been produced in simulation settings (i.e., on researchers’ well-controlled stand-alone servers and in silico studies), the application of AI models to the real world necessitates connecting to extremely complex and restricted environments involving numerous interactions, particularly with healthcare providers [6].

A five-phase process of evaluating an AI-CDSS for healthcare, from phase 0 to phase 4, has been proposed [7]. Phase 0 involves identifying users’ needs, ensuring data quality, and prototyping algorithms. Phase 1 is algorithm optimization with usability tests. Phase 2 involves measuring performance in practical real-world settings, along with measurements of clinical outcomes. Phase 3 is for identifying adverse events, and phase 4 involves post-deployment surveillance. This framework is persuasive, since its five-phase approach is widely used in drug and device evaluation process [8].

Two environments are associated with the five-phase process. First, in a simulation environment where an algorithm is produced, input variables are selected with few restrictions. If technically possible, derivative variables are liberally generated. The pool of variables is fixed and predetermined. In a real-world setting, it is emphasized that variables should be collected in economic ways, with interpretable transformations, and their time-dependent availability should be taken into consideration (Figure 2).

The second environment is the real-world setting, where it is essential to ensure that the accuracy of algorithms is maintained. The model should perform in a way that is clinically useful. In the real-world context, the output variables of algorithms have varying implications. An output variable should be selected based on its specific therapeutic value, with a focus on the difference that these prediction models could make by influencing users’ decisions and behaviors. A prediction is valuable only if it alters decisions and actions in the field.

The AI-CDSS application procedure is fundamentally dependent on multidisciplinary teams of devoted members, including experts in computer science, healthcare disciplines, and the social sciences. This team should lead the process, which consists of multiple iterations as opposed to a single intervention. It is expected that the probability of success will be increased by reliable and modern infrastructure, ongoing training, and robust communication. As mentioned above, a new era for AI-CDSS has been inaugurated, and now human intelligence will pave the way for the future.