This study set out to address the following research questions dealing with qualitative research methods within the field of HISs:

What research processes are reported in participant observation, in-depth interviews, and FGIs of hospital-based HISs and related people with respect to researcher characteristics and reflection, participant sampling, and data collection and analysis?

What are the ethical and practical dilemmas in qualitative research found in the reported research process?

We searched for qualitative research methods in HIS studies to collect articles published from 2012 to 2022 using the PubMed (MEDLINE) and CINAHL databases, referring to the COSI model, a literature search protocol for health technology assessment proposed by the U.S. National Library of Medicine [15].

The search terms were strategically made by combining three categories: types of HISs, operational terms such as “implementation” or “adaptation,” and qualitative research methods. It is useful to enter these multiple search terms because it allows for a more targeted search that matches the research objectives. It has also been demonstrated to be highly reliable, comparable to manually searching articles using a single search term [16]. Table 1 shows the search commands for merging several terms.

We found 4,607 articles in PubMed (MEDLINE) and 553 articles in CINAHL following the search strategy and identified 4,872 articles after excluding 288 duplications. After identifying the research articles, two researchers decided on inclusion/exclusion criteria in a meeting. We limited the language of our search to English. We excluded pilot research, clinical reports, policy study reports, any studies not published in peer-reviewed journals, only study abstracts published, or where the full text was unavailable. The inclusion criteria were: (1) the study is relevant to the HIS technology included in the search keywords; (2) the study reports appropriate empirical data and findings on implementing and adapting an HIS in a hospital setting; (3) the research participants must be involved in the hospital and HIS development industry; and (4) the study utilizes representative qualitative research methods (i.e., participant observation, in-depth interviews, and FGIs).

Two researchers independently reviewed the article titles and abstracts based on inclusion/exclusion criteria. After reaching a consensus, 284 articles were selected. The researchers then reviewed the full text of these articles and excluded irrelevant studies in a subsequent meeting. After this review, 74 articles were selected that matched the objective of the study, which is to investigate qualitative research methods applied in the HIS field. Figure 1 shows the process of study selection.

Two authors used the Standards for Reporting Qualitative Research (SRQR) checklist to analyze the research process of selected articles [17]. According to the SRQR developers, the checklist is a list of key factors that should be reported in utilizing qualitative research methods rather than criteria to be used to evaluate qualitative research. Therefore, we used it to determine which data to extract from the papers. To ensure the rigor of this research, we repeatedly validated the analysis while forming categories based on extracted data. We used Zotero as a bibliography management program and Microsoft Excel to organize the selected papers. A summary of the selected articles is shown in Table 2 and Appendix 1.

The characteristics of the researchers and interviewers were briefly reported in terms of qualitative research experiences or professional roles, such as doctoral students, nurse practitioners, or clinicians. A few studies reported collaborative work with medical specialists. Some articles indicated they had received additional training in qualitative research methods or HIS utilization to conduct research in hospital settings [A14, A17, A19, A28, A47, A66]. In the context of multi-disciplinary research teams engaging in HIS research, some articles reported a variety of researchers’ backgrounds, including health informatics, clinicians, nurses, and social scientists [A10, A52, A54, A70].

It was rare for authors to provide information about the researchers’ position, role, and impact on the study participants or settings. Only a few studies mentioned having no prior relationship with the research participants [A20]. Moreover, researchers rarely described the observer’s role and interactions with the participants in participant observation studies. However, they sometimes briefly reported how they intended to avoid influencing the participants during data collection and analysis. For example, researchers asked participants to confirm the collected data during interviews and observations, described procedures to minimize potential bias in data processing and analysis [A02, A05, A34, A50, A53, A54], or scheduled data collection at the participants’ convenience to avoid disrupting hospital workflows [A35].

The findings related to participant sampling, identification, and recruitment are presented in Table 5. These findings were determined by reviewing the research methods. The most common sampling method was purposive sampling, in which researchers primarily used hospital expert databases or previous research records to identify participants with specific knowledge or experience in HIS utilization or selected participants with changing characteristics and information-rich cases. At the purposefully selected study sites, some researchers utilized convenience sampling to expand the diversity of sample characteristics [A47, A65], maximum variation sampling until new information emerged [A17, A18, A24], or snowball sampling, in which participants were referred to other potential participants during observations or interviews [A04, A61, A65].

Most studies provided detailed information about the study setting, but only a few described the process of selecting the sites and the contact made with them. In cases where the identification of participants was reported, researchers identified potential participants during site visits, which was mostly done through participatory observation. Researchers visited sites as part of a preliminary survey and attended hospital meetings or workshops.

Participants in studies were recruited through various methods, such as personal invitations, emails, and telephone contacts made by researchers. Researchers used emails, flyers, and posters to encourage interested individuals to get in touch voluntarily to express their interest in participating in the study. Although researchers identified potential participants through field contact, we noticed that some studies recruited participants through direct invitations by the researchers or bulk mail sending. This was mainly because the researchers also worked as clinicians or nurses and thus fulfilled dual roles.

Table 6 presents a comprehensive review of data collection, processing, and analysis. The data were collected using one or more methods, including participatory observation, in-depth interviews, and FGIs. According to the data presented in Table 6, in-depth interviews were conducted in 29 instances, participatory observation with in-depth interviews was done in 23 studies, and FGIs were utilized in 12 studies.

Among the studies that specified the data collection process, studies that utilized participatory observation specified and pseudonymized the study site, describing the bed size, hospital staff, and brief information about the implementation and adaptation process of HISs. Studies that reported the use of participatory observation generally described the scope and interaction of observation, with some stating that they used shadowing to avoid disruption or to collect more accurate observations [A23, A31, A34, A50, A52, A64, A70].

In the studies that reported interview locations, the designated places varied, including private clinic rooms, conference rooms, and break rooms. However, the interview location was often unspecified or not stated as a private room for security or confidentiality. This suggests that data were collected by conducting informal interviews during observation or by using interview transcripts and field observation notes to answer questions as they arose.

Some studies reported conducting interviews over the phone or virtual meetings when recruiting participants for large-scale studies or due to concerns about the coronavirus disease 2019 pandemic [A01, A17, A28, A39, A67, A71]. Some studies intentionally conducted interviews during lunch or other breaks to minimize disruption [A66].

The authors of the selected papers ensured that participants’ identities were anonymized during data collection and analysis. Some authors specified how data were stored [A23, A48, A60, A64, A68, A72], but most studies did not. Only a few authors requested feedback and revisions of the manuscript from their participants [A20], considering their impact on participants. Certain studies stipluated in their ethical approval procedure that the dataset should be shared only within the research team [A40, A41, A59].

Researchers frequently utilized qualitative research software during data processing and analysis. NVivo was used in 19 of the selected papers, ATLAS.ti in 14, and Microsoft Excel in 8. There were 24 articles that did not state which software, if any, was used. All papers generally described the coding and analysis process. The most commonly reported technique to enhance trustworthiness was member checking among researchers. However, no study specified participant involvement in the review.

The rigor of qualitative research is strengthened by an explicit description of the researcher’s position and role in relation to the subjects of the study [14]. Researchers must continue to consider this requirement beyond the formal ethical approval process. However, around half of the selected articles did not provide details about the researcher or interviewer. Moreover, many clinicians or nurse researchers tended to use their workplace as the research site to conduct qualitative research in hospital settings. While this has obvious advantages for the research site, it is important to consider whether one is a researcher or a hospital worker throughout the research process. For example, if researchers are clinicians or nurses, can they remain neutral from an outsider’s perspective when investigating personal and social factors in HIS utilization? In a place that is both a workplace and a research site, can the researcher address concerns about patient care and privacy?

This also implies ethical and practical dilemmas regarding participant observation and research data collection in hospital-based research [18,19]. Formal ethical approval procedures require researchers to provide informed consent and notice to participants of their participation in the study. However, there is a practical challenge in obtaining informed consent in participatory observational research [20]. This challenge is further amplified when the research is conducted in a hospital setting, which is complex and unpredictable. For instance, in a large and busy environment like the emergency department, obtaining consent from everyone who may come under the scope of observation is extremely difficult. Furthermore, it is difficult in practice to fully explain the scope of the observation area to participants and to limit the data they collect to the scope of observations. The dilemmas associated with conducting qualitative research in the HIS field are considerable, as more than a third of the selected articles adopted participant observation as their primary research method. As a result, it is important for researchers to continually reflect on their position, role, and data collection at the beginning of a project and not just rely on considerations of the ethical approval process [19].

When researchers access medical settings for their studies, they must deal with multiple stakeholders, not just formal ethical approval. The articles that reported participant sampling generally used a purposive sampling technique, in which researchers selected or invited participants, in that the nature of HIS studies is to target participants and settings with specific expertise [12,22]. It is, therefore, important to discuss whether they have selected appropriate sampling methods. This convenience and accessibility coinciding with the purpose of the study runs up against an inevitable issue—namely, the potential for biased selection by the researcher in the process of identifying and recruiting research participants. This dilemma needs to be fully discussed in hospital-based qualitative research [22].

As we reviewed the researcher’s position and role, we noted that many clinicians and nurses decided to conduct research within their workplace. This approach allows researchers to have a dual role in a hospital setting, making it easier to identify and recruit participants. If the study aims to improve the implementation or adaptation of HISs, including examples of successes and failures may be useful. However, researchers must be clear that selecting a research setting based on accessibility is intentional [19]. It is essential to recognize that the researcher’s perceptions and decisions about accessibility can affect the research design and process of carrying out the research [23,24].

Privacy can be compromised in healthcare settings during participant recruitment and data collection. The increasing use of HIS has expanded access to personal health records and, subsequently, the number of stakeholders collecting, using, and sharing them. This has raised new ethical, privacy, and trust issues [25,26]. While discussions on patient privacy are ongoing [18,27], privacy concerns for healthcare professionals who become research participants are often overlooked. Several researchers stated that they had gone through ethical procedures regarding the privacy and confidentiality of the research participants, but most of the selected articles did not provide details on how the data from participants were managed. This may be due to researchers’ potential bias toward participants whose records exist on websites and official healthcare databases. Therefore, researchers conducting HIS research in hospital settings need to consider the confidentiality and privacy of the research participants, who are often represented as experts in specific knowledge.

In participant observation studies, researchers observe healthcare work practices related to HIS utilization. As part of this process, the researcher may observe the computer screen, which can pose a significant risk of privacy violations when recording observational data. Due to the system features of HISs, direct personal information of medical workers and patients would be coded, but the researcher could inadvertently capture medical records and notes. As discussed earlier, this is a realistic limitation of participatory observation, but it is crucial to consider the presence of such information when categorizing and writing field notes [12,19].

In conclusion, although qualitative research has been used to study hospital-based HISs and the people who use them, there needs to be more examination of researcher characteristics and reflections, participant sampling, and data collection and management in the hospital setting. Given these issues, it is essential to assess the qualitative research methods utilized in hospital-based HIS research and discuss ethical and practical considerations and issues that require researchers’ attention.